2023-02-08 02:53:46,451 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-08 02:53:46,656 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589 2023-02-08 02:53:46,671 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-08 02:53:46,672 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085, deleteOnExit=true 2023-02-08 02:53:46,672 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-08 02:53:46,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/test.cache.data in system properties and HBase conf 2023-02-08 02:53:46,673 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/hadoop.tmp.dir in system properties and HBase conf 2023-02-08 02:53:46,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/hadoop.log.dir in system properties and HBase conf 2023-02-08 02:53:46,674 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-08 02:53:46,675 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-08 02:53:46,675 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-08 02:53:46,784 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-08 02:53:47,133 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-08 02:53:47,139 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-08 02:53:47,139 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-08 02:53:47,140 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-08 02:53:47,140 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 02:53:47,140 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-08 02:53:47,141 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-08 02:53:47,141 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 02:53:47,142 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 02:53:47,142 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-08 02:53:47,142 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/nfs.dump.dir in system properties and HBase conf 2023-02-08 02:53:47,143 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/java.io.tmpdir in system properties and HBase conf 2023-02-08 02:53:47,143 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 02:53:47,144 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-08 02:53:47,144 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-08 02:53:47,608 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 02:53:47,612 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 02:53:48,192 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-08 02:53:48,343 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-08 02:53:48,356 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 02:53:48,388 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 02:53:48,415 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/java.io.tmpdir/Jetty_localhost_localdomain_36183_hdfs____.ajnjp9/webapp 2023-02-08 02:53:48,556 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:36183 2023-02-08 02:53:48,564 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 02:53:48,564 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 02:53:49,114 WARN [Listener at localhost.localdomain/36267] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 02:53:49,213 WARN [Listener at localhost.localdomain/36267] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 02:53:49,229 WARN [Listener at localhost.localdomain/36267] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 02:53:49,234 INFO [Listener at localhost.localdomain/36267] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 02:53:49,238 INFO [Listener at localhost.localdomain/36267] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/java.io.tmpdir/Jetty_localhost_40615_datanode____.by9um3/webapp 2023-02-08 02:53:49,313 INFO [Listener at localhost.localdomain/36267] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40615 2023-02-08 02:53:49,621 WARN [Listener at localhost.localdomain/45411] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 02:53:49,633 WARN [Listener at localhost.localdomain/45411] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 02:53:49,636 WARN [Listener at localhost.localdomain/45411] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 02:53:49,638 INFO [Listener at localhost.localdomain/45411] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 02:53:49,651 INFO [Listener at localhost.localdomain/45411] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/java.io.tmpdir/Jetty_localhost_41567_datanode____.pt11ts/webapp 2023-02-08 02:53:49,734 INFO [Listener at localhost.localdomain/45411] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41567 2023-02-08 02:53:49,746 WARN [Listener at localhost.localdomain/37805] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 02:53:49,762 WARN [Listener at localhost.localdomain/37805] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 02:53:49,766 WARN [Listener at localhost.localdomain/37805] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 02:53:49,768 INFO [Listener at localhost.localdomain/37805] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 02:53:49,777 INFO [Listener at localhost.localdomain/37805] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/java.io.tmpdir/Jetty_localhost_44465_datanode____4o0o2i/webapp 2023-02-08 02:53:49,888 INFO [Listener at localhost.localdomain/37805] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44465 2023-02-08 02:53:49,896 WARN [Listener at localhost.localdomain/45781] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 02:53:51,556 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcde9bfba57dbd39f: Processing first storage report for DS-244f52d0-91e1-4392-937e-393b68bc3b29 from datanode 506934cb-153f-48e7-8277-804b83b91344 2023-02-08 02:53:51,558 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcde9bfba57dbd39f: from storage DS-244f52d0-91e1-4392-937e-393b68bc3b29 node DatanodeRegistration(127.0.0.1:37075, datanodeUuid=506934cb-153f-48e7-8277-804b83b91344, infoPort=43809, infoSecurePort=0, ipcPort=45781, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,558 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8bd40f9230f6e124: Processing first storage report for DS-209e2614-3268-4b69-99fd-a68133c861e2 from datanode 8382d232-5ed5-4e55-9828-0c5294970b5d 2023-02-08 02:53:51,558 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8bd40f9230f6e124: from storage DS-209e2614-3268-4b69-99fd-a68133c861e2 node DatanodeRegistration(127.0.0.1:42397, datanodeUuid=8382d232-5ed5-4e55-9828-0c5294970b5d, infoPort=34147, infoSecurePort=0, ipcPort=45411, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb083beec1746d67a: Processing first storage report for DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d from datanode 5f50a424-dee0-448f-aec2-2d8d7ee67cbb 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb083beec1746d67a: from storage DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d node DatanodeRegistration(127.0.0.1:35289, datanodeUuid=5f50a424-dee0-448f-aec2-2d8d7ee67cbb, infoPort=35143, infoSecurePort=0, ipcPort=37805, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xcde9bfba57dbd39f: Processing first storage report for DS-4b6cce2a-bacc-4753-acfd-48f590c51c04 from datanode 506934cb-153f-48e7-8277-804b83b91344 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xcde9bfba57dbd39f: from storage DS-4b6cce2a-bacc-4753-acfd-48f590c51c04 node DatanodeRegistration(127.0.0.1:37075, datanodeUuid=506934cb-153f-48e7-8277-804b83b91344, infoPort=43809, infoSecurePort=0, ipcPort=45781, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x8bd40f9230f6e124: Processing first storage report for DS-838393cf-f916-4025-b6cd-477cc7a6fa9c from datanode 8382d232-5ed5-4e55-9828-0c5294970b5d 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x8bd40f9230f6e124: from storage DS-838393cf-f916-4025-b6cd-477cc7a6fa9c node DatanodeRegistration(127.0.0.1:42397, datanodeUuid=8382d232-5ed5-4e55-9828-0c5294970b5d, infoPort=34147, infoSecurePort=0, ipcPort=45411, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,559 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb083beec1746d67a: Processing first storage report for DS-981afeef-02e5-4c97-92c9-a2597cc7f0f7 from datanode 5f50a424-dee0-448f-aec2-2d8d7ee67cbb 2023-02-08 02:53:51,560 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb083beec1746d67a: from storage DS-981afeef-02e5-4c97-92c9-a2597cc7f0f7 node DatanodeRegistration(127.0.0.1:35289, datanodeUuid=5f50a424-dee0-448f-aec2-2d8d7ee67cbb, infoPort=35143, infoSecurePort=0, ipcPort=37805, storageInfo=lv=-57;cid=testClusterID;nsid=723205654;c=1675824827679), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 02:53:51,646 DEBUG [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589 2023-02-08 02:53:51,701 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/zookeeper_0, clientPort=65244, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-08 02:53:51,712 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=65244 2023-02-08 02:53:51,720 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:51,721 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:52,351 INFO [Listener at localhost.localdomain/45781] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621 with version=8 2023-02-08 02:53:52,351 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/hbase-staging 2023-02-08 02:53:52,656 INFO [Listener at localhost.localdomain/45781] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-08 02:53:53,050 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): master/jenkins-hbase12:0 server-side Connection retries=45 2023-02-08 02:53:53,077 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,078 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,078 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 02:53:53,078 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,079 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 02:53:53,208 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 02:53:53,267 DEBUG [Listener at localhost.localdomain/45781] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-08 02:53:53,345 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /136.243.104.168:39013 2023-02-08 02:53:53,354 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,355 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,373 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39013 connecting to ZooKeeper ensemble=127.0.0.1:65244 2023-02-08 02:53:53,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:390130x0, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 02:53:53,622 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39013-0x101407d68340000 connected 2023-02-08 02:53:53,711 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 02:53:53,712 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:53:53,716 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 02:53:53,723 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39013 2023-02-08 02:53:53,723 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39013 2023-02-08 02:53:53,724 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39013 2023-02-08 02:53:53,724 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39013 2023-02-08 02:53:53,725 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39013 2023-02-08 02:53:53,730 INFO [Listener at localhost.localdomain/45781] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621, hbase.cluster.distributed=false 2023-02-08 02:53:53,788 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-08 02:53:53,789 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,789 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,789 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 02:53:53,789 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,789 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 02:53:53,793 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 02:53:53,796 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /136.243.104.168:46783 2023-02-08 02:53:53,798 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 02:53:53,804 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 02:53:53,805 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,807 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,808 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46783 connecting to ZooKeeper ensemble=127.0.0.1:65244 2023-02-08 02:53:53,821 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:467830x0, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 02:53:53,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46783-0x101407d68340001 connected 2023-02-08 02:53:53,823 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:467830x0, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 02:53:53,824 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:53:53,825 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 02:53:53,826 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46783 2023-02-08 02:53:53,828 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46783 2023-02-08 02:53:53,828 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46783 2023-02-08 02:53:53,828 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46783 2023-02-08 02:53:53,828 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46783 2023-02-08 02:53:53,839 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-08 02:53:53,839 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,839 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,840 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 02:53:53,840 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,840 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 02:53:53,840 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 02:53:53,844 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /136.243.104.168:36075 2023-02-08 02:53:53,844 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 02:53:53,845 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 02:53:53,846 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,848 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,850 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36075 connecting to ZooKeeper ensemble=127.0.0.1:65244 2023-02-08 02:53:53,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:360750x0, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 02:53:53,865 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36075-0x101407d68340002 connected 2023-02-08 02:53:53,865 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 02:53:53,866 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:53:53,867 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 02:53:53,867 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36075 2023-02-08 02:53:53,867 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36075 2023-02-08 02:53:53,869 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36075 2023-02-08 02:53:53,869 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36075 2023-02-08 02:53:53,869 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36075 2023-02-08 02:53:53,880 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase12:0 server-side Connection retries=45 2023-02-08 02:53:53,880 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,880 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,880 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 02:53:53,881 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 02:53:53,881 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 02:53:53,881 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 02:53:53,884 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /136.243.104.168:37605 2023-02-08 02:53:53,884 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 02:53:53,889 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 02:53:53,891 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,893 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:53,894 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37605 connecting to ZooKeeper ensemble=127.0.0.1:65244 2023-02-08 02:53:53,906 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:376050x0, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 02:53:53,907 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:376050x0, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 02:53:53,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37605-0x101407d68340003 connected 2023-02-08 02:53:53,908 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:53:53,909 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 02:53:53,909 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37605 2023-02-08 02:53:53,909 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37605 2023-02-08 02:53:53,912 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37605 2023-02-08 02:53:53,912 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37605 2023-02-08 02:53:53,913 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37605 2023-02-08 02:53:53,914 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:53,931 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 02:53:53,933 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:53,962 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 02:53:53,962 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 02:53:53,962 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 02:53:53,962 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 02:53:53,964 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:53,964 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 02:53:53,965 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase12.apache.org,39013,1675824832471 from backup master directory 2023-02-08 02:53:53,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 02:53:53,979 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:53,979 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 02:53:53,980 WARN [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 02:53:53,980 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:53,982 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-08 02:53:53,984 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-08 02:53:54,070 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/hbase.id with ID: be7ac225-4d66-4c91-a974-5488885e6616 2023-02-08 02:53:54,108 INFO [master/jenkins-hbase12:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 02:53:54,131 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:54,176 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x74ecf107 to 127.0.0.1:65244 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 02:53:54,214 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5fe58222, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 02:53:54,236 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 02:53:54,238 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-08 02:53:54,254 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-08 02:53:54,254 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-08 02:53:54,256 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:53:54,260 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:53:54,262 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 02:53:54,292 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store-tmp 2023-02-08 02:53:54,366 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:54,367 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 02:53:54,367 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:53:54,367 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:53:54,367 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 02:53:54,367 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:53:54,367 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:53:54,367 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675824834366Disabling compacts and flushes for region at 1675824834367 (+1 ms)Disabling writes for close at 1675824834367Writing region close event to WAL at 1675824834367Closed at 1675824834367 2023-02-08 02:53:54,369 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/WALs/jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:54,390 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C39013%2C1675824832471, suffix=, logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/WALs/jenkins-hbase12.apache.org,39013,1675824832471, archiveDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/oldWALs, maxLogs=10 2023-02-08 02:53:54,446 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK] 2023-02-08 02:53:54,446 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK] 2023-02-08 02:53:54,446 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK] 2023-02-08 02:53:54,457 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:53:54,531 INFO [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/WALs/jenkins-hbase12.apache.org,39013,1675824832471/jenkins-hbase12.apache.org%2C39013%2C1675824832471.1675824834399 2023-02-08 02:53:54,532 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK], DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK], DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK]] 2023-02-08 02:53:54,532 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:53:54,532 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:54,535 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,536 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,587 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,594 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-08 02:53:54,625 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-08 02:53:54,639 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:54,644 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,646 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,660 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 02:53:54,664 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:53:54,665 INFO [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73555380, jitterRate=0.09606057405471802}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 02:53:54,666 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675824834559Initializing all the Stores at 1675824834560 (+1 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824834561 (+1 ms)Cleaning up temporary data from old regions at 1675824834652 (+91 ms)Cleaning up detritus from prior splits at 1675824834652Region opened successfully at 1675824834666 (+14 ms) 2023-02-08 02:53:54,667 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-08 02:53:54,686 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:53:54,686 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:53:54,688 INFO [master/jenkins-hbase12:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:53:54,690 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:53:54,717 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 26 msec 2023-02-08 02:53:54,717 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-08 02:53:54,739 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-08 02:53:54,744 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-08 02:53:54,766 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-08 02:53:54,770 INFO [master/jenkins-hbase12:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-08 02:53:54,771 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-08 02:53:54,775 INFO [master/jenkins-hbase12:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-08 02:53:54,779 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-08 02:53:54,825 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:54,826 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-08 02:53:54,827 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-08 02:53:54,839 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-08 02:53:54,853 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 02:53:54,853 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 02:53:54,853 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 02:53:54,853 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 02:53:54,853 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:54,853 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase12.apache.org,39013,1675824832471, sessionid=0x101407d68340000, setting cluster-up flag (Was=false) 2023-02-08 02:53:54,884 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:54,916 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-08 02:53:54,917 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:54,941 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:54,973 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-08 02:53:54,974 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:54,976 WARN [master/jenkins-hbase12:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.hbase-snapshot/.tmp 2023-02-08 02:53:55,016 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(952): ClusterId : be7ac225-4d66-4c91-a974-5488885e6616 2023-02-08 02:53:55,016 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(952): ClusterId : be7ac225-4d66-4c91-a974-5488885e6616 2023-02-08 02:53:55,016 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(952): ClusterId : be7ac225-4d66-4c91-a974-5488885e6616 2023-02-08 02:53:55,022 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 02:53:55,022 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 02:53:55,022 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 02:53:55,048 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 02:53:55,048 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 02:53:55,048 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 02:53:55,048 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 02:53:55,048 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 02:53:55,048 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 02:53:55,068 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 02:53:55,068 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 02:53:55,068 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 02:53:55,070 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ReadOnlyZKClient(139): Connect 0x2afdd777 to 127.0.0.1:65244 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 02:53:55,070 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ReadOnlyZKClient(139): Connect 0x17fe48f2 to 127.0.0.1:65244 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 02:53:55,070 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ReadOnlyZKClient(139): Connect 0x66db461f to 127.0.0.1:65244 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 02:53:55,082 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-08 02:53:55,091 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-08 02:53:55,091 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=5, maxPoolSize=5 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase12:0, corePoolSize=10, maxPoolSize=10 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-08 02:53:55,092 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,093 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824865093 2023-02-08 02:53:55,094 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-08 02:53:55,098 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 02:53:55,099 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-08 02:53:55,099 DEBUG [RS:2;jenkins-hbase12:37605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@243a2c58, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 02:53:55,100 DEBUG [RS:2;jenkins-hbase12:37605] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2005bc07, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-08 02:53:55,100 DEBUG [RS:0;jenkins-hbase12:46783] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@681ca4b4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 02:53:55,100 DEBUG [RS:0;jenkins-hbase12:46783] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c232d6f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-08 02:53:55,101 DEBUG [RS:1;jenkins-hbase12:36075] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1939a47d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 02:53:55,101 DEBUG [RS:1;jenkins-hbase12:36075] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d062382, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-08 02:53:55,104 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 02:53:55,106 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-08 02:53:55,115 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-08 02:53:55,115 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-08 02:53:55,116 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-08 02:53:55,116 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-08 02:53:55,119 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,121 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-08 02:53:55,122 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-08 02:53:55,122 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-08 02:53:55,123 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase12:37605 2023-02-08 02:53:55,125 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase12:46783 2023-02-08 02:53:55,125 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-08 02:53:55,126 INFO [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-08 02:53:55,126 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase12:36075 2023-02-08 02:53:55,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675824835127,5,FailOnTimeoutGroup] 2023-02-08 02:53:55,127 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675824835127,5,FailOnTimeoutGroup] 2023-02-08 02:53:55,127 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,127 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-08 02:53:55,128 INFO [RS:1;jenkins-hbase12:36075] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 02:53:55,129 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,129 INFO [RS:1;jenkins-hbase12:36075] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 02:53:55,128 INFO [RS:0;jenkins-hbase12:46783] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 02:53:55,129 INFO [RS:0;jenkins-hbase12:46783] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 02:53:55,129 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,128 INFO [RS:2;jenkins-hbase12:37605] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 02:53:55,129 INFO [RS:2;jenkins-hbase12:37605] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 02:53:55,129 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 02:53:55,129 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 02:53:55,129 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 02:53:55,132 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39013,1675824832471 with isa=jenkins-hbase12.apache.org/136.243.104.168:37605, startcode=1675824833879 2023-02-08 02:53:55,132 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39013,1675824832471 with isa=jenkins-hbase12.apache.org/136.243.104.168:46783, startcode=1675824833788 2023-02-08 02:53:55,132 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase12.apache.org,39013,1675824832471 with isa=jenkins-hbase12.apache.org/136.243.104.168:36075, startcode=1675824833838 2023-02-08 02:53:55,162 DEBUG [RS:0;jenkins-hbase12:46783] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 02:53:55,162 DEBUG [RS:2;jenkins-hbase12:37605] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 02:53:55,164 DEBUG [RS:1;jenkins-hbase12:36075] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 02:53:55,165 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 02:53:55,166 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 02:53:55,167 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621 2023-02-08 02:53:55,204 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:55,208 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 02:53:55,212 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/info 2023-02-08 02:53:55,213 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 02:53:55,215 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,215 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 02:53:55,218 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/rep_barrier 2023-02-08 02:53:55,218 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 02:53:55,220 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,220 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 02:53:55,223 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/table 2023-02-08 02:53:55,224 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 02:53:55,225 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,227 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740 2023-02-08 02:53:55,228 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740 2023-02-08 02:53:55,233 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 02:53:55,234 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:55713, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 02:53:55,234 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:50901, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 02:53:55,234 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:54651, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 02:53:55,236 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 02:53:55,240 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:53:55,241 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70020590, jitterRate=0.04338809847831726}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 02:53:55,241 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675824835206Initializing all the Stores at 1675824835207 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675824835207Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824835208 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675824835208Cleaning up temporary data from old regions at 1675824835230 (+22 ms)Cleaning up detritus from prior splits at 1675824835231 (+1 ms)Region opened successfully at 1675824835241 (+10 ms) 2023-02-08 02:53:55,243 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 02:53:55,243 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 02:53:55,243 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 02:53:55,243 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 02:53:55,243 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 02:53:55,249 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39013] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,250 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 02:53:55,251 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.ServerManager(394): Registering regionserver=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,251 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675824835243Disabling compacts and flushes for region at 1675824835243Disabling writes for close at 1675824835243Writing region close event to WAL at 1675824835250 (+7 ms)Closed at 1675824835250 2023-02-08 02:53:55,257 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 02:53:55,257 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-08 02:53:55,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-08 02:53:55,273 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621 2023-02-08 02:53:55,273 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621 2023-02-08 02:53:55,273 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621 2023-02-08 02:53:55,273 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36267 2023-02-08 02:53:55,273 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36267 2023-02-08 02:53:55,273 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 02:53:55,273 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36267 2023-02-08 02:53:55,273 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 02:53:55,273 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 02:53:55,287 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-08 02:53:55,289 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 02:53:55,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:53:55,337 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ZKUtil(162): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,337 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ZKUtil(162): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,337 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ZKUtil(162): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,337 WARN [RS:2;jenkins-hbase12:37605] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 02:53:55,338 WARN [RS:1;jenkins-hbase12:36075] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 02:53:55,337 WARN [RS:0;jenkins-hbase12:46783] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 02:53:55,339 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,37605,1675824833879] 2023-02-08 02:53:55,339 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,46783,1675824833788] 2023-02-08 02:53:55,339 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase12.apache.org,36075,1675824833838] 2023-02-08 02:53:55,339 INFO [RS:0;jenkins-hbase12:46783] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 02:53:55,338 INFO [RS:1;jenkins-hbase12:36075] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 02:53:55,338 INFO [RS:2;jenkins-hbase12:37605] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 02:53:55,339 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,339 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,339 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,354 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ZKUtil(162): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,354 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ZKUtil(162): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,354 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ZKUtil(162): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,354 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ZKUtil(162): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,354 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ZKUtil(162): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,354 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ZKUtil(162): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,355 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ZKUtil(162): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,355 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ZKUtil(162): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,355 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ZKUtil(162): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,364 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 02:53:55,364 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 02:53:55,364 DEBUG [RS:1;jenkins-hbase12:36075] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 02:53:55,373 INFO [RS:2;jenkins-hbase12:37605] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 02:53:55,374 INFO [RS:1;jenkins-hbase12:36075] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 02:53:55,373 INFO [RS:0;jenkins-hbase12:46783] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 02:53:55,393 INFO [RS:2;jenkins-hbase12:37605] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 02:53:55,393 INFO [RS:0;jenkins-hbase12:46783] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 02:53:55,393 INFO [RS:1;jenkins-hbase12:36075] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 02:53:55,398 INFO [RS:0;jenkins-hbase12:46783] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 02:53:55,398 INFO [RS:1;jenkins-hbase12:36075] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 02:53:55,398 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,398 INFO [RS:2;jenkins-hbase12:37605] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 02:53:55,398 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,399 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,399 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 02:53:55,399 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 02:53:55,399 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 02:53:55,406 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,406 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,406 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,406 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,406 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,407 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:2;jenkins-hbase12:37605] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:0;jenkins-hbase12:46783] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase12:0, corePoolSize=2, maxPoolSize=2 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,408 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,409 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,409 DEBUG [RS:1;jenkins-hbase12:36075] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase12:0, corePoolSize=1, maxPoolSize=1 2023-02-08 02:53:55,413 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,413 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,413 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,413 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,413 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,413 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,417 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,417 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,417 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,427 INFO [RS:2;jenkins-hbase12:37605] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 02:53:55,428 INFO [RS:1;jenkins-hbase12:36075] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 02:53:55,429 INFO [RS:0;jenkins-hbase12:46783] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 02:53:55,431 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,36075,1675824833838-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,431 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,46783,1675824833788-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,431 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,37605,1675824833879-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,441 DEBUG [jenkins-hbase12:39013] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-08 02:53:55,446 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:53:55,453 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:53:55,453 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:53:55,453 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:53:55,453 INFO [RS:2;jenkins-hbase12:37605] regionserver.Replication(203): jenkins-hbase12.apache.org,37605,1675824833879 started 2023-02-08 02:53:55,453 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:53:55,453 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,37605,1675824833879, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:37605, sessionid=0x101407d68340003 2023-02-08 02:53:55,457 INFO [RS:1;jenkins-hbase12:36075] regionserver.Replication(203): jenkins-hbase12.apache.org,36075,1675824833838 started 2023-02-08 02:53:55,457 INFO [RS:0;jenkins-hbase12:46783] regionserver.Replication(203): jenkins-hbase12.apache.org,46783,1675824833788 started 2023-02-08 02:53:55,457 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,36075,1675824833838, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:36075, sessionid=0x101407d68340002 2023-02-08 02:53:55,457 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 02:53:55,457 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1638): Serving as jenkins-hbase12.apache.org,46783,1675824833788, RpcServer on jenkins-hbase12.apache.org/136.243.104.168:46783, sessionid=0x101407d68340001 2023-02-08 02:53:55,458 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 02:53:55,458 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 02:53:55,458 DEBUG [RS:0;jenkins-hbase12:46783] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,458 DEBUG [RS:2;jenkins-hbase12:37605] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,458 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,46783,1675824833788' 2023-02-08 02:53:55,458 DEBUG [RS:1;jenkins-hbase12:36075] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,459 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:55,458 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,37605,1675824833879' 2023-02-08 02:53:55,459 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,36075,1675824833838' 2023-02-08 02:53:55,459 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:55,459 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:55,461 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,46783,1675824833788, state=OPENING 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:55,465 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:55,465 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,46783,1675824833788' 2023-02-08 02:53:55,465 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 02:53:55,465 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 02:53:55,466 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 02:53:55,466 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 02:53:55,466 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 02:53:55,466 DEBUG [RS:2;jenkins-hbase12:37605] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:55,466 DEBUG [RS:0;jenkins-hbase12:46783] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 02:53:55,466 DEBUG [RS:1;jenkins-hbase12:36075] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:55,467 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,37605,1675824833879' 2023-02-08 02:53:55,467 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 02:53:55,467 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase12.apache.org,36075,1675824833838' 2023-02-08 02:53:55,467 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 02:53:55,467 DEBUG [RS:0;jenkins-hbase12:46783] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 02:53:55,467 INFO [RS:0;jenkins-hbase12:46783] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 02:53:55,467 INFO [RS:0;jenkins-hbase12:46783] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 02:53:55,467 DEBUG [RS:1;jenkins-hbase12:36075] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 02:53:55,468 DEBUG [RS:1;jenkins-hbase12:36075] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 02:53:55,468 INFO [RS:1;jenkins-hbase12:36075] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 02:53:55,468 INFO [RS:1;jenkins-hbase12:36075] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 02:53:55,470 DEBUG [RS:2;jenkins-hbase12:37605] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 02:53:55,470 DEBUG [RS:2;jenkins-hbase12:37605] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 02:53:55,470 INFO [RS:2;jenkins-hbase12:37605] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 02:53:55,470 INFO [RS:2;jenkins-hbase12:37605] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 02:53:55,478 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-08 02:53:55,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:55,489 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 02:53:55,492 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:53:55,576 INFO [RS:1;jenkins-hbase12:36075] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C36075%2C1675824833838, suffix=, logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,36075,1675824833838, archiveDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs, maxLogs=32 2023-02-08 02:53:55,576 INFO [RS:2;jenkins-hbase12:37605] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C37605%2C1675824833879, suffix=, logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,37605,1675824833879, archiveDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs, maxLogs=32 2023-02-08 02:53:55,576 INFO [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C46783%2C1675824833788, suffix=, logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788, archiveDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs, maxLogs=32 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK] 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK] 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK] 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK] 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK] 2023-02-08 02:53:55,605 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK] 2023-02-08 02:53:55,618 INFO [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788/jenkins-hbase12.apache.org%2C46783%2C1675824833788.1675824835580 2023-02-08 02:53:55,626 DEBUG [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK], DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK], DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK]] 2023-02-08 02:53:55,626 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK] 2023-02-08 02:53:55,627 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK] 2023-02-08 02:53:55,630 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK] 2023-02-08 02:53:55,630 INFO [RS:1;jenkins-hbase12:36075] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,36075,1675824833838/jenkins-hbase12.apache.org%2C36075%2C1675824833838.1675824835580 2023-02-08 02:53:55,633 DEBUG [RS:1;jenkins-hbase12:36075] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK], DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK], DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK]] 2023-02-08 02:53:55,638 INFO [RS:2;jenkins-hbase12:37605] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,37605,1675824833879/jenkins-hbase12.apache.org%2C37605%2C1675824833879.1675824835580 2023-02-08 02:53:55,639 DEBUG [RS:2;jenkins-hbase12:37605] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK], DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK], DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK]] 2023-02-08 02:53:55,671 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:55,673 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 02:53:55,677 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 02:53:55,688 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-08 02:53:55,689 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 02:53:55,693 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase12.apache.org%2C46783%2C1675824833788.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788, archiveDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs, maxLogs=32 2023-02-08 02:53:55,713 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK] 2023-02-08 02:53:55,713 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK] 2023-02-08 02:53:55,717 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK] 2023-02-08 02:53:55,721 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788/jenkins-hbase12.apache.org%2C46783%2C1675824833788.meta.1675824835694.meta 2023-02-08 02:53:55,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42397,DS-209e2614-3268-4b69-99fd-a68133c861e2,DISK], DatanodeInfoWithStorage[127.0.0.1:37075,DS-244f52d0-91e1-4392-937e-393b68bc3b29,DISK], DatanodeInfoWithStorage[127.0.0.1:35289,DS-89cd2a99-b70f-4dd5-977e-bd1fd842ad4d,DISK]] 2023-02-08 02:53:55,723 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:53:55,727 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:53:55,728 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-08 02:53:55,741 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-08 02:53:55,744 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-08 02:53:55,747 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-08 02:53:55,748 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:55,748 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-08 02:53:55,748 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-08 02:53:55,750 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 02:53:55,753 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/info 2023-02-08 02:53:55,754 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/info 2023-02-08 02:53:55,755 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 02:53:55,757 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,758 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 02:53:55,759 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/rep_barrier 2023-02-08 02:53:55,759 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/rep_barrier 2023-02-08 02:53:55,760 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 02:53:55,761 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,762 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 02:53:55,764 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/table 2023-02-08 02:53:55,764 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/table 2023-02-08 02:53:55,764 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 02:53:55,765 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:55,768 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740 2023-02-08 02:53:55,771 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740 2023-02-08 02:53:55,775 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 02:53:55,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 02:53:55,779 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68299759, jitterRate=0.017745718359947205}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 02:53:55,779 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675824835748Writing region info on filesystem at 1675824835748Initializing all the Stores at 1675824835750 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675824835750Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824835750Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675824835750Cleaning up temporary data from old regions at 1675824835773 (+23 ms)Cleaning up detritus from prior splits at 1675824835774 (+1 ms)Running coprocessor post-open hooks at 1675824835779 (+5 ms)Region opened successfully at 1675824835779 2023-02-08 02:53:55,788 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675824835665 2023-02-08 02:53:55,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-08 02:53:55,806 INFO [RS_OPEN_META-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-08 02:53:55,807 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase12.apache.org,46783,1675824833788, state=OPEN 2023-02-08 02:53:55,821 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-08 02:53:55,821 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 02:53:55,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-08 02:53:55,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase12.apache.org,46783,1675824833788 in 330 msec 2023-02-08 02:53:55,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-08 02:53:55,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 559 msec 2023-02-08 02:53:55,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 821 msec 2023-02-08 02:53:55,839 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675824833950, completionTime=-1 2023-02-08 02:53:55,839 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-08 02:53:55,839 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:53:55,897 DEBUG [hconnection-0x196478f2-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 02:53:55,900 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48098, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 02:53:55,915 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:53:55,916 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824895916 2023-02-08 02:53:55,916 INFO [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824955916 2023-02-08 02:53:55,916 INFO [master/jenkins-hbase12:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 76 msec 2023-02-08 02:53:55,962 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39013,1675824832471-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,962 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39013,1675824832471-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,962 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39013,1675824832471-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,965 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase12:39013, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,965 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-08 02:53:55,972 DEBUG [master/jenkins-hbase12:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-08 02:53:55,985 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-08 02:53:55,987 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 02:53:55,996 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-08 02:53:55,998 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:53:56,001 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:53:56,024 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,026 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5 empty. 2023-02-08 02:53:56,027 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,027 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-08 02:53:56,068 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-08 02:53:56,070 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1dababf94e1d1ac62f98f642609de9f5, NAME => 'hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:53:56,091 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:56,091 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 1dababf94e1d1ac62f98f642609de9f5, disabling compactions & flushes 2023-02-08 02:53:56,091 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,091 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,091 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. after waiting 0 ms 2023-02-08 02:53:56,092 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,092 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,092 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 1dababf94e1d1ac62f98f642609de9f5: Waiting for close lock at 1675824836091Disabling compacts and flushes for region at 1675824836091Disabling writes for close at 1675824836092 (+1 ms)Writing region close event to WAL at 1675824836092Closed at 1675824836092 2023-02-08 02:53:56,097 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:53:56,111 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675824836100"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824836100"}]},"ts":"1675824836100"} 2023-02-08 02:53:56,144 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:53:56,147 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:53:56,151 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824836147"}]},"ts":"1675824836147"} 2023-02-08 02:53:56,156 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-08 02:53:56,180 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:53:56,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:53:56,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:53:56,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:53:56,181 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:53:56,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1dababf94e1d1ac62f98f642609de9f5, ASSIGN}] 2023-02-08 02:53:56,189 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=1dababf94e1d1ac62f98f642609de9f5, ASSIGN 2023-02-08 02:53:56,191 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=1dababf94e1d1ac62f98f642609de9f5, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:53:56,344 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:53:56,345 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1dababf94e1d1ac62f98f642609de9f5, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:56,345 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675824836344"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824836344"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824836344"}]},"ts":"1675824836344"} 2023-02-08 02:53:56,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 1dababf94e1d1ac62f98f642609de9f5, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:53:56,504 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:56,504 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 02:53:56,508 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:46592, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 02:53:56,515 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,516 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1dababf94e1d1ac62f98f642609de9f5, NAME => 'hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:53:56,519 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:53:56,519 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,520 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:56,520 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,520 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,522 INFO [StoreOpener-1dababf94e1d1ac62f98f642609de9f5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,524 DEBUG [StoreOpener-1dababf94e1d1ac62f98f642609de9f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/info 2023-02-08 02:53:56,524 DEBUG [StoreOpener-1dababf94e1d1ac62f98f642609de9f5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/info 2023-02-08 02:53:56,525 INFO [StoreOpener-1dababf94e1d1ac62f98f642609de9f5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1dababf94e1d1ac62f98f642609de9f5 columnFamilyName info 2023-02-08 02:53:56,526 INFO [StoreOpener-1dababf94e1d1ac62f98f642609de9f5-1] regionserver.HStore(310): Store=1dababf94e1d1ac62f98f642609de9f5/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:56,528 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,530 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,534 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:53:56,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:53:56,539 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 1dababf94e1d1ac62f98f642609de9f5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70738474, jitterRate=0.05408540368080139}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 02:53:56,539 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 1dababf94e1d1ac62f98f642609de9f5: Running coprocessor pre-open hook at 1675824836520Writing region info on filesystem at 1675824836520Initializing all the Stores at 1675824836521 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675824836522 (+1 ms)Cleaning up temporary data from old regions at 1675824836530 (+8 ms)Cleaning up detritus from prior splits at 1675824836531 (+1 ms)Running coprocessor post-open hooks at 1675824836539 (+8 ms)Region opened successfully at 1675824836539 2023-02-08 02:53:56,541 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5., pid=6, masterSystemTime=1675824836504 2023-02-08 02:53:56,546 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,547 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:53:56,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:56,549 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675824836548"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824836548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824836548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824836548"}]},"ts":"1675824836548"} 2023-02-08 02:53:56,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-08 02:53:56,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 1dababf94e1d1ac62f98f642609de9f5, server=jenkins-hbase12.apache.org,37605,1675824833879 in 204 msec 2023-02-08 02:53:56,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-08 02:53:56,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=1dababf94e1d1ac62f98f642609de9f5, ASSIGN in 373 msec 2023-02-08 02:53:56,563 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:53:56,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824836563"}]},"ts":"1675824836563"} 2023-02-08 02:53:56,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-08 02:53:56,616 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-08 02:53:56,617 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:53:56,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 629 msec 2023-02-08 02:53:56,625 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-08 02:53:56,625 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:56,644 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 02:53:56,646 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:46606, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 02:53:56,658 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-08 02:53:56,684 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 02:53:56,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 48 msec 2023-02-08 02:53:56,712 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:53:56,730 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 02:53:56,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 33 msec 2023-02-08 02:53:56,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-08 02:53:56,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-08 02:53:56,802 INFO [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.822sec 2023-02-08 02:53:56,805 INFO [master/jenkins-hbase12:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-08 02:53:56,806 INFO [master/jenkins-hbase12:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-08 02:53:56,806 INFO [master/jenkins-hbase12:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-08 02:53:56,808 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39013,1675824832471-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-08 02:53:56,810 INFO [master/jenkins-hbase12:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase12.apache.org,39013,1675824832471-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-08 02:53:56,846 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-08 02:53:56,849 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(139): Connect 0x140ff023 to 127.0.0.1:65244 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 02:53:56,864 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7c6a47b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 02:53:56,877 DEBUG [hconnection-0x187a2033-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 02:53:56,889 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:48112, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 02:53:56,898 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:53:56,927 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=362, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=173, AvailableMemoryMB=3861 2023-02-08 02:53:56,931 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:53:56,932 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:56,932 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:56,939 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 02:53:56,942 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:58546, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 02:53:56,972 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:53:56,972 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:53:56,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:53:56,979 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:53:56,981 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:53:56,984 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:53:56,986 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:56,987 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 empty. 2023-02-08 02:53:56,990 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:56,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 02:53:57,010 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 02:53:57,012 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 61b3f9480f07089e1b2877d5f55bdd46, NAME => 'testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:53:57,034 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:57,034 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 61b3f9480f07089e1b2877d5f55bdd46, disabling compactions & flushes 2023-02-08 02:53:57,034 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,034 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,035 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. after waiting 0 ms 2023-02-08 02:53:57,035 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,035 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,035 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 61b3f9480f07089e1b2877d5f55bdd46: Waiting for close lock at 1675824837034Disabling compacts and flushes for region at 1675824837034Disabling writes for close at 1675824837035 (+1 ms)Writing region close event to WAL at 1675824837035Closed at 1675824837035 2023-02-08 02:53:57,039 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:53:57,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824837041"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824837041"}]},"ts":"1675824837041"} 2023-02-08 02:53:57,043 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:53:57,045 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:53:57,045 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824837045"}]},"ts":"1675824837045"} 2023-02-08 02:53:57,048 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-08 02:53:57,068 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:53:57,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:53:57,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:53:57,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:53:57,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:53:57,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, ASSIGN}] 2023-02-08 02:53:57,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, ASSIGN 2023-02-08 02:53:57,074 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:53:57,224 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:53:57,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=61b3f9480f07089e1b2877d5f55bdd46, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824837225"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824837225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824837225"}]},"ts":"1675824837225"} 2023-02-08 02:53:57,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 61b3f9480f07089e1b2877d5f55bdd46, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:53:57,383 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,383 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 02:53:57,386 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:36766, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 02:53:57,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 61b3f9480f07089e1b2877d5f55bdd46, NAME => 'testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:53:57,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:53:57,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:53:57,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,395 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,397 DEBUG [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1 2023-02-08 02:53:57,397 DEBUG [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1 2023-02-08 02:53:57,397 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 61b3f9480f07089e1b2877d5f55bdd46 columnFamilyName cf1 2023-02-08 02:53:57,398 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] regionserver.HStore(310): Store=61b3f9480f07089e1b2877d5f55bdd46/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:57,399 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,401 DEBUG [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2 2023-02-08 02:53:57,401 DEBUG [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2 2023-02-08 02:53:57,401 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 61b3f9480f07089e1b2877d5f55bdd46 columnFamilyName cf2 2023-02-08 02:53:57,402 INFO [StoreOpener-61b3f9480f07089e1b2877d5f55bdd46-1] regionserver.HStore(310): Store=61b3f9480f07089e1b2877d5f55bdd46/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:53:57,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:53:57,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:57,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:53:57,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 61b3f9480f07089e1b2877d5f55bdd46; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72256027, jitterRate=0.07669870555400848}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:53:57,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 61b3f9480f07089e1b2877d5f55bdd46: Running coprocessor pre-open hook at 1675824837393Writing region info on filesystem at 1675824837393Initializing all the Stores at 1675824837394 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824837394Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824837394Cleaning up temporary data from old regions at 1675824837408 (+14 ms)Cleaning up detritus from prior splits at 1675824837409 (+1 ms)Running coprocessor post-open hooks at 1675824837416 (+7 ms)Region opened successfully at 1675824837416 2023-02-08 02:53:57,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46., pid=11, masterSystemTime=1675824837383 2023-02-08 02:53:57,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=61b3f9480f07089e1b2877d5f55bdd46, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824837423"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824837423"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824837423"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824837423"}]},"ts":"1675824837423"} 2023-02-08 02:53:57,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-08 02:53:57,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 61b3f9480f07089e1b2877d5f55bdd46, server=jenkins-hbase12.apache.org,36075,1675824833838 in 198 msec 2023-02-08 02:53:57,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-08 02:53:57,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, ASSIGN in 361 msec 2023-02-08 02:53:57,436 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:53:57,437 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824837436"}]},"ts":"1675824837436"} 2023-02-08 02:53:57,439 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-08 02:53:57,447 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:53:57,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 471 msec 2023-02-08 02:53:57,484 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 02:53:57,490 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:36774, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 02:53:57,506 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:57,506 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:57,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:57,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,517 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:57,517 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,518 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:57,518 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:57,519 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,519 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,530 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,530 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,530 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,530 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,531 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,531 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,534 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,534 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,534 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,534 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:57,535 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:57,537 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:57,537 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,537 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:57,537 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:57,538 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:53:57,547 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,547 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,548 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,548 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,548 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:57,548 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:57,606 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/735b5af7179c496f8245ee4a69f43afb 2023-02-08 02:53:57,667 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/89320c6927e2470b87fb071cd3a4af67 2023-02-08 02:53:57,679 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/735b5af7179c496f8245ee4a69f43afb as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb 2023-02-08 02:53:57,690 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:53:57,693 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/89320c6927e2470b87fb071cd3a4af67 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67 2023-02-08 02:53:57,702 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:53:57,704 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 166ms, sequenceid=6, compaction requested=false 2023-02-08 02:53:57,706 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:57,708 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:57,708 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:57,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:57,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:57,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:57,720 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:57,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:57,720 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,720 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:57,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:57,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:57,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:57,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:57,724 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:57,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44a7792c[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:57,724 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,762 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,762 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,762 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:57,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:57,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:57,763 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:57,763 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:57,764 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:57,764 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:57,784 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:57,784 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,784 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:57,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:57,785 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:57,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:57,794 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:57,794 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:57,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:57,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:57,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:57,799 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:57,799 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:57,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:57,799 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:57,799 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,799 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:57,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:57,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,811 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,812 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,812 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:57,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,814 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,815 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,815 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,835 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:57,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,836 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:57,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,837 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,837 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:57,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,837 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,837 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,837 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:57,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:57,838 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:57,838 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:57,838 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,838 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,839 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:57,839 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:57,939 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:57,940 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:57,940 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:57,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:57,973 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:57,973 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:57,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:57,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:53:57,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,978 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:57,978 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,979 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:57,979 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:57,979 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,979 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,989 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,989 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,989 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,989 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:57,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,990 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:57,990 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,990 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,991 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,991 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,991 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,992 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,992 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:57,992 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:57,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:57,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,993 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,993 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,993 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,994 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:57,994 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:57,996 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:57,996 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:57,996 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:57,996 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:57,996 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:57,997 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:57,997 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:57,997 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:57,997 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:57,997 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:53:58,004 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,014 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,015 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,015 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,016 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/659477cbc6424dc0b3bd5589973ce32a 2023-02-08 02:53:58,020 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,021 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,043 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/7e9fcc80761045548beeb0a859436c4e 2023-02-08 02:53:58,052 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/659477cbc6424dc0b3bd5589973ce32a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a 2023-02-08 02:53:58,061 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:53:58,062 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/7e9fcc80761045548beeb0a859436c4e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e 2023-02-08 02:53:58,072 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:53:58,073 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 76ms, sequenceid=19, compaction requested=false 2023-02-08 02:53:58,073 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:58,074 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:58,074 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:58,074 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:58,074 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,074 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,074 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,084 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,084 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,084 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,084 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56575428[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:58,087 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:58,087 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,099 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:58,099 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,100 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,100 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:58,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,110 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,130 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,130 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,130 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:58,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:58,135 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:58,135 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:58,135 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:58,135 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,135 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:58,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,148 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,149 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,151 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,151 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,151 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:58,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,185 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,186 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,186 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,186 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,187 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,287 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:58,287 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,287 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:58,313 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,313 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:58,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:53:58,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,317 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,318 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:58,318 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:58,318 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,318 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,337 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,337 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,337 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,337 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,337 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,339 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,339 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,339 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,339 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,339 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,339 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,339 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,340 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:58,340 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,340 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:58,341 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,341 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,341 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,341 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:53:58,351 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,362 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,362 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,362 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,362 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,362 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,365 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/1403d96eb8654f26a685d444bc1c43a5 2023-02-08 02:53:58,387 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/1780f1bd5a964e379239b226093a5243 2023-02-08 02:53:58,396 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/1403d96eb8654f26a685d444bc1c43a5 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5 2023-02-08 02:53:58,405 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:53:58,406 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/1780f1bd5a964e379239b226093a5243 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243 2023-02-08 02:53:58,416 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:53:58,418 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 77ms, sequenceid=32, compaction requested=true 2023-02-08 02:53:58,418 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:58,418 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:58,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:58,419 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:58,419 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,419 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,419 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,431 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,432 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,432 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:58,434 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c6b13f1[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:58,434 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:58,434 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,446 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,446 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:58,446 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,446 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:58,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:58,447 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:58,458 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,458 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,458 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,467 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,467 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,467 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,477 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,478 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,478 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,477 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:58,483 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:58,483 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:58,483 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:58,483 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:58,483 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,483 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,495 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,496 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,496 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,498 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,498 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,499 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,510 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,510 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,510 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:58,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,510 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,511 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:58,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,512 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,511 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:58,512 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,512 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,612 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:58,612 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,612 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:58,638 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,638 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:58,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:53:58,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,641 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,641 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,642 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:58,642 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:58,643 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,643 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,652 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,653 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,653 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,654 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,654 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,654 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:58,655 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,655 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,655 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,655 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:58,656 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,656 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:58,656 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:58,656 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:53:58,667 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,674 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/79da4e17085a4deaa91dd2d3a8bdf775 2023-02-08 02:53:58,677 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,677 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,678 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,678 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,678 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,696 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/4c20e078b8c44f70b7df56c103d5acb6 2023-02-08 02:53:58,705 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/79da4e17085a4deaa91dd2d3a8bdf775 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775 2023-02-08 02:53:58,713 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:53:58,715 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/4c20e078b8c44f70b7df56c103d5acb6 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6 2023-02-08 02:53:58,723 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:53:58,724 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 68ms, sequenceid=45, compaction requested=true 2023-02-08 02:53:58,725 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:58,725 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:58,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:58,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:58,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:58,726 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:58,785 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,786 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,786 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:58,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:58,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69adf4aa[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:58,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:58,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,811 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:58,811 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:58,812 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:58,821 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,821 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,821 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,831 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,832 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,832 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,831 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,831 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:58,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,832 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,833 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:58,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:58,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:58,837 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:58,837 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:58,837 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:58,837 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,837 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:58,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,847 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,846 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:58,847 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,847 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:58,849 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,849 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,849 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,863 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,863 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,864 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:58,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,864 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,865 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:58,865 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:58,865 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,865 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:58,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:58,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:58,865 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,965 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:58,965 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,966 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:58,991 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:58,991 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:58,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:58,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:53:58,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,995 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:58,995 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:58,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:58,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:58,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:58,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,060 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,060 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:59,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,062 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,062 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,062 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,062 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,062 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,062 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,065 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,066 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,066 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,067 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,067 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,068 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:59,068 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:59,068 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,068 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:59,068 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:53:59,086 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/c47420784748405b940d977e8883aa99 2023-02-08 02:53:59,094 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,108 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,108 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,108 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,111 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab 2023-02-08 02:53:59,120 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/c47420784748405b940d977e8883aa99 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99 2023-02-08 02:53:59,129 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:53:59,130 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab 2023-02-08 02:53:59,138 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:53:59,140 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 72ms, sequenceid=58, compaction requested=true 2023-02-08 02:53:59,140 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:59,140 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,140 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:59,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:59,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,151 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,151 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:59,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16bc9eaa[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:59,154 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:59,154 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:59,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,162 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,162 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,163 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:59,162 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,163 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,163 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:59,172 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,172 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,172 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,183 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,183 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,183 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,193 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,193 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,194 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:59,200 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:59,200 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:59,200 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:59,200 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:59,200 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,200 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,211 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,210 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,211 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,212 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,213 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,214 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,214 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:59,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,250 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,250 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:59,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,251 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,351 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:59,351 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:59,352 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:59,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:59,377 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:59,378 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:59,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:53:59,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:53:59,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,381 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:59,381 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,382 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-08 02:53:59,382 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-08 02:53:59,383 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,383 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,410 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,410 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,410 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,410 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,411 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,411 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,412 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,415 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,416 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,416 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,416 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-08 02:53:59,416 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,416 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,416 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,416 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,416 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,416 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,416 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,417 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,416 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-08 02:53:59,417 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:53:59,417 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,417 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:53:59,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:53:59,418 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,418 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. started... 2023-02-08 02:53:59,419 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:53:59,425 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,438 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/d630463c1a1147a28c1ab28bc4d39442 2023-02-08 02:53:59,459 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/3605855ab6eb43789d09e46ce1ef4ce1 2023-02-08 02:53:59,467 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/d630463c1a1147a28c1ab28bc4d39442 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442 2023-02-08 02:53:59,475 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:53:59,477 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/3605855ab6eb43789d09e46ce1ef4ce1 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1 2023-02-08 02:53:59,485 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:53:59,486 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 67ms, sequenceid=71, compaction requested=true 2023-02-08 02:53:59,486 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 61b3f9480f07089e1b2877d5f55bdd46: 2023-02-08 02:53:59,486 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,487 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:53:59,487 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:53:59,487 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,487 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-08 02:53:59,487 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-08 02:53:59,499 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,499 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,499 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,499 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:53:59,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-08 02:53:59,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a266e41[Count = 0] remaining members to acquire global barrier 2023-02-08 02:53:59,502 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-08 02:53:59,502 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,510 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:53:59,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,509 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:53:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:53:59,510 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:53:59,520 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,520 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,520 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,530 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,530 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,530 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-08 02:53:59,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-08 02:53:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:53:59,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:53:59,545 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-08 02:53:59,545 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:53:59,545 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:53:59,545 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,545 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,558 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,558 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:53:59,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,559 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,559 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,561 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:53:59,561 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,561 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:53:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-08 02:53:59,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:53:59,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:53:59,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,573 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,573 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:53:59,573 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:53:59,573 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:53:59,574 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:53:59,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,573 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:53:59,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:53:59,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:53:59,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:53:59,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-08 02:53:59,674 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:53:59,674 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:53:59,674 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:53:59,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:53:59,699 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,705 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:53:59,706 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:53:59,706 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:53:59,706 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:53:59,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, UNASSIGN}] 2023-02-08 02:53:59,715 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, UNASSIGN 2023-02-08 02:53:59,716 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=61b3f9480f07089e1b2877d5f55bdd46, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:53:59,717 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824839716"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824839716"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824839716"}]},"ts":"1675824839716"} 2023-02-08 02:53:59,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 61b3f9480f07089e1b2877d5f55bdd46, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:53:59,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:59,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 61b3f9480f07089e1b2877d5f55bdd46, disabling compactions & flushes 2023-02-08 02:53:59,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. after waiting 0 ms 2023-02-08 02:53:59,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 61b3f9480f07089e1b2877d5f55bdd46 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:53:59,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/804ee8f8e6654569ac69368576da6e9a 2023-02-08 02:53:59,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/feb7567280fb4a639a9e34e73d880c03 2023-02-08 02:53:59,930 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf1/804ee8f8e6654569ac69368576da6e9a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a 2023-02-08 02:53:59,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:53:59,940 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.tmp/cf2/feb7567280fb4a639a9e34e73d880c03 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03 2023-02-08 02:53:59,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:53:59,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 71ms, sequenceid=82, compaction requested=true 2023-02-08 02:53:59,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:53:59,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:53:59,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. 2023-02-08 02:53:59,970 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 61b3f9480f07089e1b2877d5f55bdd46: Waiting for close lock at 1675824839879Running coprocessor pre-close hooks at 1675824839879Disabling compacts and flushes for region at 1675824839879Disabling writes for close at 1675824839879Obtaining lock to block concurrent updates at 1675824839879Preparing flush snapshotting stores in 61b3f9480f07089e1b2877d5f55bdd46 at 1675824839879Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824839880 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. at 1675824839880Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf1: creating writer at 1675824839881 (+1 ms)Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf1: appending metadata at 1675824839884 (+3 ms)Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf1: closing flushed file at 1675824839884Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf2: creating writer at 1675824839907 (+23 ms)Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf2: appending metadata at 1675824839911 (+4 ms)Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf2: closing flushed file at 1675824839911Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf1: reopening flushed file at 1675824839931 (+20 ms)Flushing 61b3f9480f07089e1b2877d5f55bdd46/cf2: reopening flushed file at 1675824839942 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 61b3f9480f07089e1b2877d5f55bdd46 in 71ms, sequenceid=82, compaction requested=true at 1675824839950 (+8 ms)Writing region close event to WAL at 1675824839964 (+14 ms)Running coprocessor post-close hooks at 1675824839969 (+5 ms)Closed at 1675824839970 (+1 ms) 2023-02-08 02:53:59,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:59,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=61b3f9480f07089e1b2877d5f55bdd46, regionState=CLOSED 2023-02-08 02:53:59,974 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824839973"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824839973"}]},"ts":"1675824839973"} 2023-02-08 02:53:59,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-08 02:53:59,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 61b3f9480f07089e1b2877d5f55bdd46, server=jenkins-hbase12.apache.org,36075,1675824833838 in 257 msec 2023-02-08 02:53:59,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-08 02:53:59,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=61b3f9480f07089e1b2877d5f55bdd46, UNASSIGN in 265 msec 2023-02-08 02:53:59,998 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=61b3f9480f07089e1b2877d5f55bdd46, threads=14 2023-02-08 02:53:59,999 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:59,999 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:59,999 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:53:59,999 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,000 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,000 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,001 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,001 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,002 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,005 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,005 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,005 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,005 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,006 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,037 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1, top=true 2023-02-08 02:54:00,053 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a, top=true 2023-02-08 02:54:00,054 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab, top=true 2023-02-08 02:54:00,055 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03, top=true 2023-02-08 02:54:00,091 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99, top=true 2023-02-08 02:54:00,093 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442, top=true 2023-02-08 02:54:00,111 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,111 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,119 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1 for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,119 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,133 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,138 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99 for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,129 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03 for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,146 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,133 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,151 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/.splits/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442 for child: 7e254f706c3041846348609ee0d53549, parent: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,156 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,146 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,183 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,217 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,218 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,224 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,227 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,234 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6 for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,234 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,236 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a for region: 61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:00,237 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 61b3f9480f07089e1b2877d5f55bdd46 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:00,291 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:00,294 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:00,297 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675824840297"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675824840297"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675824840297"}]},"ts":"1675824840297"} 2023-02-08 02:54:00,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840297"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824840297"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824840297"}]},"ts":"1675824840297"} 2023-02-08 02:54:00,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840297"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824840297"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824840297"}]},"ts":"1675824840297"} 2023-02-08 02:54:00,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, ASSIGN}] 2023-02-08 02:54:00,347 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, ASSIGN 2023-02-08 02:54:00,348 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:54:00,349 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, ASSIGN 2023-02-08 02:54:00,350 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:54:00,498 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:54:00,500 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7e254f706c3041846348609ee0d53549, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:00,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840499"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824840499"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824840499"}]},"ts":"1675824840499"} 2023-02-08 02:54:00,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 7e254f706c3041846348609ee0d53549, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:00,507 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c97fd6e93be08968b8a2ca2719084f7a, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:00,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840507"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824840507"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824840507"}]},"ts":"1675824840507"} 2023-02-08 02:54:00,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure c97fd6e93be08968b8a2ca2719084f7a, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:00,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c97fd6e93be08968b8a2ca2719084f7a, NAME => 'testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.', STARTKEY => '', ENDKEY => '26'} 2023-02-08 02:54:00,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:00,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:00,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,663 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,664 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1 2023-02-08 02:54:00,664 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1 2023-02-08 02:54:00,665 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c97fd6e93be08968b8a2ca2719084f7a columnFamilyName cf1 2023-02-08 02:54:00,702 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-bottom 2023-02-08 02:54:00,712 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-bottom 2023-02-08 02:54:00,721 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-bottom 2023-02-08 02:54:00,730 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-bottom 2023-02-08 02:54:00,731 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(310): Store=c97fd6e93be08968b8a2ca2719084f7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:00,731 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,733 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2 2023-02-08 02:54:00,733 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2 2023-02-08 02:54:00,733 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c97fd6e93be08968b8a2ca2719084f7a columnFamilyName cf2 2023-02-08 02:54:00,749 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-bottom 2023-02-08 02:54:00,768 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-bottom 2023-02-08 02:54:00,780 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-bottom 2023-02-08 02:54:00,800 DEBUG [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-bottom 2023-02-08 02:54:00,800 INFO [StoreOpener-c97fd6e93be08968b8a2ca2719084f7a-1] regionserver.HStore(310): Store=c97fd6e93be08968b8a2ca2719084f7a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:00,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:00,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:00,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened c97fd6e93be08968b8a2ca2719084f7a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59353683, jitterRate=-0.11556120216846466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:00,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for c97fd6e93be08968b8a2ca2719084f7a: Running coprocessor pre-open hook at 1675824840661Writing region info on filesystem at 1675824840661Initializing all the Stores at 1675824840662 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824840662Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824840663 (+1 ms)Cleaning up temporary data from old regions at 1675824840807 (+144 ms)Cleaning up detritus from prior splits at 1675824840808 (+1 ms)Running coprocessor post-open hooks at 1675824840814 (+6 ms)Region opened successfully at 1675824840814 2023-02-08 02:54:00,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a., pid=18, masterSystemTime=1675824840655 2023-02-08 02:54:00,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:00,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:00,819 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 02:54:00,819 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 02:54:00,823 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1912): c97fd6e93be08968b8a2ca2719084f7a/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:00,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1912): c97fd6e93be08968b8a2ca2719084f7a/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:00,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c97fd6e93be08968b8a2ca2719084f7a/cf2 in testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c97fd6e93be08968b8a2ca2719084f7a/cf1 in testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:00,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e254f706c3041846348609ee0d53549, NAME => 'testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.', STARTKEY => '26', ENDKEY => ''} 2023-02-08 02:54:00,823 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/.tmp, totalSize=19.9 K 2023-02-08 02:54:00,824 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/.tmp, totalSize=19.9 K 2023-02-08 02:54:00,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:00,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:00,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=c97fd6e93be08968b8a2ca2719084f7a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:00,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840825"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824840825"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824840825"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824840825"}]},"ts":"1675824840825"} 2023-02-08 02:54:00,826 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824837491 2023-02-08 02:54:00,826 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824837491 2023-02-08 02:54:00,828 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824837947 2023-02-08 02:54:00,828 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824837947 2023-02-08 02:54:00,829 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,829 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824838290 2023-02-08 02:54:00,830 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824838290 2023-02-08 02:54:00,830 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824838614 2023-02-08 02:54:00,830 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824838614 2023-02-08 02:54:00,832 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1 2023-02-08 02:54:00,832 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1 2023-02-08 02:54:00,833 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e254f706c3041846348609ee0d53549 columnFamilyName cf1 2023-02-08 02:54:00,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-08 02:54:00,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure c97fd6e93be08968b8a2ca2719084f7a, server=jenkins-hbase12.apache.org,36075,1675824833838 in 320 msec 2023-02-08 02:54:00,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, ASSIGN in 491 msec 2023-02-08 02:54:00,861 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] throttle.PressureAwareThroughputController(145): c97fd6e93be08968b8a2ca2719084f7a#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:00,864 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] throttle.PressureAwareThroughputController(145): c97fd6e93be08968b8a2ca2719084f7a#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:00,864 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-top 2023-02-08 02:54:00,874 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-top 2023-02-08 02:54:00,892 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-top 2023-02-08 02:54:00,900 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-top 2023-02-08 02:54:00,903 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/.tmp/cf1/73b5a204340f45f991a5f465c56b1186 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/73b5a204340f45f991a5f465c56b1186 2023-02-08 02:54:00,904 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/.tmp/cf2/f72547c75d5c43dba3b3d9abb1b5862d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/f72547c75d5c43dba3b3d9abb1b5862d 2023-02-08 02:54:00,907 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a 2023-02-08 02:54:00,916 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99 2023-02-08 02:54:00,918 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c97fd6e93be08968b8a2ca2719084f7a/cf1 of c97fd6e93be08968b8a2ca2719084f7a into 73b5a204340f45f991a5f465c56b1186(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:00,918 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c97fd6e93be08968b8a2ca2719084f7a/cf2 of c97fd6e93be08968b8a2ca2719084f7a into f72547c75d5c43dba3b3d9abb1b5862d(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:00,919 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c97fd6e93be08968b8a2ca2719084f7a: 2023-02-08 02:54:00,919 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c97fd6e93be08968b8a2ca2719084f7a: 2023-02-08 02:54:00,919 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a., storeName=c97fd6e93be08968b8a2ca2719084f7a/cf1, priority=12, startTime=1675824840818; duration=0sec 2023-02-08 02:54:00,919 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a., storeName=c97fd6e93be08968b8a2ca2719084f7a/cf2, priority=12, startTime=1675824840819; duration=0sec 2023-02-08 02:54:00,919 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:00,919 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:00,921 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442 2023-02-08 02:54:00,922 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(310): Store=7e254f706c3041846348609ee0d53549/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:00,922 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,923 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2 2023-02-08 02:54:00,923 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2 2023-02-08 02:54:00,924 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e254f706c3041846348609ee0d53549 columnFamilyName cf2 2023-02-08 02:54:00,939 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-top 2023-02-08 02:54:00,945 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-top 2023-02-08 02:54:00,952 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-top 2023-02-08 02:54:00,958 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-top 2023-02-08 02:54:00,964 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab 2023-02-08 02:54:00,969 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1 2023-02-08 02:54:00,978 DEBUG [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03 2023-02-08 02:54:00,978 INFO [StoreOpener-7e254f706c3041846348609ee0d53549-1] regionserver.HStore(310): Store=7e254f706c3041846348609ee0d53549/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:00,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:00,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:00,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 7e254f706c3041846348609ee0d53549; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68998986, jitterRate=0.02816501259803772}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:00,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 7e254f706c3041846348609ee0d53549: Running coprocessor pre-open hook at 1675824840826Writing region info on filesystem at 1675824840826Initializing all the Stores at 1675824840828 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824840828Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824840829 (+1 ms)Cleaning up temporary data from old regions at 1675824840981 (+152 ms)Cleaning up detritus from prior splits at 1675824840981Running coprocessor post-open hooks at 1675824840985 (+4 ms)Region opened successfully at 1675824840985 2023-02-08 02:54:00,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549., pid=17, masterSystemTime=1675824840655 2023-02-08 02:54:00,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:00,987 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:00,987 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:00,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:00,987 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,987 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,987 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1912): 7e254f706c3041846348609ee0d53549/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:00,987 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1912): 7e254f706c3041846348609ee0d53549/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:00,988 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e254f706c3041846348609ee0d53549/cf1 in testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,988 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e254f706c3041846348609ee0d53549/cf2 in testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,988 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/.tmp, totalSize=34.9 K 2023-02-08 02:54:00,988 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/.tmp, totalSize=34.9 K 2023-02-08 02:54:00,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,990 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824837491 2023-02-08 02:54:00,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:00,990 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824837491 2023-02-08 02:54:00,991 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=7e254f706c3041846348609ee0d53549, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:00,991 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824837947 2023-02-08 02:54:00,991 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824840991"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824840991"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824840991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824840991"}]},"ts":"1675824840991"} 2023-02-08 02:54:00,992 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824837947 2023-02-08 02:54:00,992 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824838290 2023-02-08 02:54:00,993 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824838290 2023-02-08 02:54:00,993 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824838614 2023-02-08 02:54:00,994 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824838614 2023-02-08 02:54:00,994 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824838968 2023-02-08 02:54:00,994 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824838968 2023-02-08 02:54:00,994 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824839354 2023-02-08 02:54:00,995 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824839354 2023-02-08 02:54:00,995 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824839677 2023-02-08 02:54:00,995 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824839677 2023-02-08 02:54:01,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-08 02:54:01,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 7e254f706c3041846348609ee0d53549, server=jenkins-hbase12.apache.org,36075,1675824833838 in 496 msec 2023-02-08 02:54:01,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-08 02:54:01,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, ASSIGN in 659 msec 2023-02-08 02:54:01,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 in 1.3110 sec 2023-02-08 02:54:01,028 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] throttle.PressureAwareThroughputController(145): 7e254f706c3041846348609ee0d53549#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:01,030 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e254f706c3041846348609ee0d53549#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:01,043 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/.tmp/cf1/99a86803544a400f9b626d291fa9253d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/99a86803544a400f9b626d291fa9253d 2023-02-08 02:54:01,045 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/.tmp/cf2/f304b6b91e4f4d27a36095681ea5e5cf as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/f304b6b91e4f4d27a36095681ea5e5cf 2023-02-08 02:54:01,050 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e254f706c3041846348609ee0d53549/cf1 of 7e254f706c3041846348609ee0d53549 into 99a86803544a400f9b626d291fa9253d(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:01,050 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e254f706c3041846348609ee0d53549: 2023-02-08 02:54:01,050 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549., storeName=7e254f706c3041846348609ee0d53549/cf1, priority=9, startTime=1675824840986; duration=0sec 2023-02-08 02:54:01,050 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:01,052 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7e254f706c3041846348609ee0d53549/cf2 of 7e254f706c3041846348609ee0d53549 into f304b6b91e4f4d27a36095681ea5e5cf(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:01,053 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e254f706c3041846348609ee0d53549: 2023-02-08 02:54:01,053 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549., storeName=7e254f706c3041846348609ee0d53549/cf2, priority=9, startTime=1675824840987; duration=0sec 2023-02-08 02:54:01,053 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:01,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36075] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:36774 deadline: 1675824901207, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46. is not online on jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:01,300 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:01,342 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 02:54:01,367 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 02:54:01,370 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-08 02:54:01,388 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-08 02:54:01,390 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-08 02:54:01,416 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:01,417 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:01,417 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:01,428 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:01,428 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:01,429 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-08 02:54:01,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionUnevenDaughter 2023-02-08 02:54:01,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:01,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824841447"}]},"ts":"1675824841447"} 2023-02-08 02:54:01,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-08 02:54:01,451 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:01,451 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:01,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 02:54:01,509 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-08 02:54:01,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, UNASSIGN}] 2023-02-08 02:54:01,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, UNASSIGN 2023-02-08 02:54:01,514 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7e254f706c3041846348609ee0d53549, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:01,514 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824841513"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824841513"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824841513"}]},"ts":"1675824841513"} 2023-02-08 02:54:01,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 7e254f706c3041846348609ee0d53549, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:01,519 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, UNASSIGN 2023-02-08 02:54:01,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c97fd6e93be08968b8a2ca2719084f7a, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:01,521 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824841520"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824841520"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824841520"}]},"ts":"1675824841520"} 2023-02-08 02:54:01,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure c97fd6e93be08968b8a2ca2719084f7a, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:01,557 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:01,557 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:01,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 02:54:01,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:01,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 7e254f706c3041846348609ee0d53549, disabling compactions & flushes 2023-02-08 02:54:01,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:01,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:01,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. after waiting 0 ms 2023-02-08 02:54:01,669 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:01,679 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a] to archive 2023-02-08 02:54:01,681 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:01,686 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,688 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,690 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,691 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,693 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-c47420784748405b940d977e8883aa99 2023-02-08 02:54:01,694 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-d630463c1a1147a28c1ab28bc4d39442 2023-02-08 02:54:01,696 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-804ee8f8e6654569ac69368576da6e9a 2023-02-08 02:54:01,712 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03] to archive 2023-02-08 02:54:01,713 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:01,715 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,717 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,718 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,720 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,721 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-1cab3c62457a4ad2b2fbb815d9c3fdab 2023-02-08 02:54:01,723 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-3605855ab6eb43789d09e46ce1ef4ce1 2023-02-08 02:54:01,724 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/testSplitTableRegionUnevenDaughter=61b3f9480f07089e1b2877d5f55bdd46-feb7567280fb4a639a9e34e73d880c03 2023-02-08 02:54:01,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:54:01,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:01,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549. 2023-02-08 02:54:01,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 7e254f706c3041846348609ee0d53549: Waiting for close lock at 1675824841669Running coprocessor pre-close hooks at 1675824841669Disabling compacts and flushes for region at 1675824841669Disabling writes for close at 1675824841669Writing region close event to WAL at 1675824841725 (+56 ms)Running coprocessor post-close hooks at 1675824841730 (+5 ms)Closed at 1675824841730 2023-02-08 02:54:01,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7e254f706c3041846348609ee0d53549 2023-02-08 02:54:01,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:01,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing c97fd6e93be08968b8a2ca2719084f7a, disabling compactions & flushes 2023-02-08 02:54:01,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:01,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:01,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. after waiting 0 ms 2023-02-08 02:54:01,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:01,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=7e254f706c3041846348609ee0d53549, regionState=CLOSED 2023-02-08 02:54:01,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824841733"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824841733"}]},"ts":"1675824841733"} 2023-02-08 02:54:01,734 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775-bottom] to archive 2023-02-08 02:54:01,735 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:01,737 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/735b5af7179c496f8245ee4a69f43afb.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,740 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/659477cbc6424dc0b3bd5589973ce32a.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-08 02:54:01,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 7e254f706c3041846348609ee0d53549, server=jenkins-hbase12.apache.org,36075,1675824833838 in 222 msec 2023-02-08 02:54:01,743 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/1403d96eb8654f26a685d444bc1c43a5.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e254f706c3041846348609ee0d53549, UNASSIGN in 230 msec 2023-02-08 02:54:01,745 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/79da4e17085a4deaa91dd2d3a8bdf775.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,747 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6-bottom] to archive 2023-02-08 02:54:01,748 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:01,749 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/89320c6927e2470b87fb071cd3a4af67.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,751 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/7e9fcc80761045548beeb0a859436c4e.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,752 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/1780f1bd5a964e379239b226093a5243.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,753 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/4c20e078b8c44f70b7df56c103d5acb6.61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:01,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:54:01,758 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:01,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:01,759 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:01,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a. 2023-02-08 02:54:01,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for c97fd6e93be08968b8a2ca2719084f7a: Waiting for close lock at 1675824841733Running coprocessor pre-close hooks at 1675824841733Disabling compacts and flushes for region at 1675824841733Disabling writes for close at 1675824841733Writing region close event to WAL at 1675824841755 (+22 ms)Running coprocessor post-close hooks at 1675824841759 (+4 ms)Closed at 1675824841760 (+1 ms) 2023-02-08 02:54:01,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 02:54:01,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:01,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c97fd6e93be08968b8a2ca2719084f7a, regionState=CLOSED 2023-02-08 02:54:01,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675824841763"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824841763"}]},"ts":"1675824841763"} 2023-02-08 02:54:01,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-08 02:54:01,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure c97fd6e93be08968b8a2ca2719084f7a, server=jenkins-hbase12.apache.org,36075,1675824833838 in 242 msec 2023-02-08 02:54:01,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-08 02:54:01,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=c97fd6e93be08968b8a2ca2719084f7a, UNASSIGN in 256 msec 2023-02-08 02:54:01,770 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824841770"}]},"ts":"1675824841770"} 2023-02-08 02:54:01,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-08 02:54:02,010 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-08 02:54:02,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 576 msec 2023-02-08 02:54:02,061 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,061 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 02:54:02,063 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-08 02:54:02,064 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,064 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,075 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,077 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,077 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 02:54:02,085 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:02,085 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549 2023-02-08 02:54:02,085 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:02,090 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/recovered.edits] 2023-02-08 02:54:02,090 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/recovered.edits] 2023-02-08 02:54:02,090 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/recovered.edits] 2023-02-08 02:54:02,093 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/73b5a204340f45f991a5f465c56b1186 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf1/73b5a204340f45f991a5f465c56b1186 2023-02-08 02:54:02,093 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/1403d96eb8654f26a685d444bc1c43a5 2023-02-08 02:54:02,093 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/99a86803544a400f9b626d291fa9253d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf1/99a86803544a400f9b626d291fa9253d 2023-02-08 02:54:02,095 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/659477cbc6424dc0b3bd5589973ce32a 2023-02-08 02:54:02,096 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/f72547c75d5c43dba3b3d9abb1b5862d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/cf2/f72547c75d5c43dba3b3d9abb1b5862d 2023-02-08 02:54:02,096 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/f304b6b91e4f4d27a36095681ea5e5cf to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/cf2/f304b6b91e4f4d27a36095681ea5e5cf 2023-02-08 02:54:02,098 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/735b5af7179c496f8245ee4a69f43afb 2023-02-08 02:54:02,099 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/79da4e17085a4deaa91dd2d3a8bdf775 2023-02-08 02:54:02,099 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a/recovered.edits/90.seqid 2023-02-08 02:54:02,099 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549/recovered.edits/90.seqid 2023-02-08 02:54:02,100 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/c97fd6e93be08968b8a2ca2719084f7a 2023-02-08 02:54:02,100 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e254f706c3041846348609ee0d53549 2023-02-08 02:54:02,101 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/804ee8f8e6654569ac69368576da6e9a 2023-02-08 02:54:02,102 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/c47420784748405b940d977e8883aa99 2023-02-08 02:54:02,103 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf1/d630463c1a1147a28c1ab28bc4d39442 2023-02-08 02:54:02,106 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1780f1bd5a964e379239b226093a5243 2023-02-08 02:54:02,107 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/1cab3c62457a4ad2b2fbb815d9c3fdab 2023-02-08 02:54:02,109 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/3605855ab6eb43789d09e46ce1ef4ce1 2023-02-08 02:54:02,111 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/4c20e078b8c44f70b7df56c103d5acb6 2023-02-08 02:54:02,112 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/7e9fcc80761045548beeb0a859436c4e 2023-02-08 02:54:02,114 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/89320c6927e2470b87fb071cd3a4af67 2023-02-08 02:54:02,115 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/cf2/feb7567280fb4a639a9e34e73d880c03 2023-02-08 02:54:02,118 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46/recovered.edits/85.seqid 2023-02-08 02:54:02,119 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionUnevenDaughter/61b3f9480f07089e1b2877d5f55bdd46 2023-02-08 02:54:02,119 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-08 02:54:02,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,141 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-08 02:54:02,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-08 02:54:02,179 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,179 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 02:54:02,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-08 02:54:02,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824842180"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:02,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824842180"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:02,181 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824842180"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:02,184 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:54:02,184 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 61b3f9480f07089e1b2877d5f55bdd46, NAME => 'testSplitTableRegionUnevenDaughter,,1675824836977.61b3f9480f07089e1b2877d5f55bdd46.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c97fd6e93be08968b8a2ca2719084f7a, NAME => 'testSplitTableRegionUnevenDaughter,,1675824839699.c97fd6e93be08968b8a2ca2719084f7a.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 7e254f706c3041846348609ee0d53549, NAME => 'testSplitTableRegionUnevenDaughter,26,1675824839699.7e254f706c3041846348609ee0d53549.', STARTKEY => '26', ENDKEY => ''}] 2023-02-08 02:54:02,184 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-08 02:54:02,184 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824842184"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:02,186 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-08 02:54:02,194 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:02,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 126 msec 2023-02-08 02:54:02,381 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,381 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-08 02:54:02,382 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-08 02:54:02,398 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=395 (was 362) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:51812 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53852 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57296 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57208 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54160 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:52052 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53826 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54176 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53656 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54166 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase12:36075-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57328 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53724 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:52042 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53876 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57160 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54128 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=837 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=259 (was 279), ProcessCount=173 (was 173), AvailableMemoryMB=3815 (was 3861) 2023-02-08 02:54:02,411 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395, OpenFileDescriptor=837, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=173, AvailableMemoryMB=3814 2023-02-08 02:54:02,412 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:02,412 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:02,412 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:02,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:54:02,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:54:02,518 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:02,520 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:54:02,521 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:54:02,523 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,524 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c empty. 2023-02-08 02:54:02,525 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,525 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 02:54:02,536 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:02,537 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => edae7e6dcabc9beb6fca230c3a7df74c, NAME => 'testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing edae7e6dcabc9beb6fca230c3a7df74c, disabling compactions & flushes 2023-02-08 02:54:02,548 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. after waiting 0 ms 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,548 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,548 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for edae7e6dcabc9beb6fca230c3a7df74c: Waiting for close lock at 1675824842548Disabling compacts and flushes for region at 1675824842548Disabling writes for close at 1675824842548Writing region close event to WAL at 1675824842548Closed at 1675824842548 2023-02-08 02:54:02,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:54:02,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824842552"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824842552"}]},"ts":"1675824842552"} 2023-02-08 02:54:02,554 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:54:02,555 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:54:02,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824842555"}]},"ts":"1675824842555"} 2023-02-08 02:54:02,557 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-08 02:54:02,589 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:02,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:02,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:02,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:02,591 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:54:02,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, ASSIGN}] 2023-02-08 02:54:02,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, ASSIGN 2023-02-08 02:54:02,594 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:54:02,652 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 02:54:02,653 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-08 02:54:02,653 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:02,653 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-08 02:54:02,744 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:02,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:02,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824842745"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824842745"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824842745"}]},"ts":"1675824842745"} 2023-02-08 02:54:02,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:02,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => edae7e6dcabc9beb6fca230c3a7df74c, NAME => 'testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:02,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:02,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:02,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,907 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,908 DEBUG [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1 2023-02-08 02:54:02,909 DEBUG [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1 2023-02-08 02:54:02,909 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region edae7e6dcabc9beb6fca230c3a7df74c columnFamilyName cf1 2023-02-08 02:54:02,910 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] regionserver.HStore(310): Store=edae7e6dcabc9beb6fca230c3a7df74c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:02,910 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,911 DEBUG [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2 2023-02-08 02:54:02,911 DEBUG [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2 2023-02-08 02:54:02,912 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region edae7e6dcabc9beb6fca230c3a7df74c columnFamilyName cf2 2023-02-08 02:54:02,914 INFO [StoreOpener-edae7e6dcabc9beb6fca230c3a7df74c-1] regionserver.HStore(310): Store=edae7e6dcabc9beb6fca230c3a7df74c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:02,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:02,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:02,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:54:02,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened edae7e6dcabc9beb6fca230c3a7df74c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58767630, jitterRate=-0.12429407238960266}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:02,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for edae7e6dcabc9beb6fca230c3a7df74c: Running coprocessor pre-open hook at 1675824842906Writing region info on filesystem at 1675824842906Initializing all the Stores at 1675824842907 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824842907Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824842907Cleaning up temporary data from old regions at 1675824842916 (+9 ms)Cleaning up detritus from prior splits at 1675824842916Running coprocessor post-open hooks at 1675824842922 (+6 ms)Region opened successfully at 1675824842922 2023-02-08 02:54:02,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c., pid=27, masterSystemTime=1675824842901 2023-02-08 02:54:02,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:02,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:02,926 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824842926"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824842926"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824842926"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824842926"}]},"ts":"1675824842926"} 2023-02-08 02:54:02,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-08 02:54:02,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 in 180 msec 2023-02-08 02:54:02,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-08 02:54:02,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, ASSIGN in 340 msec 2023-02-08 02:54:02,935 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:54:02,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824842935"}]},"ts":"1675824842935"} 2023-02-08 02:54:02,937 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-08 02:54:02,948 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:54:02,950 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 431 msec 2023-02-08 02:54:03,030 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,030 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,031 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:03,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:03,033 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:03,033 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,034 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:03,034 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:03,035 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,035 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,046 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,046 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,047 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,048 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,048 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,048 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,048 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,048 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,049 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,050 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:03,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:03,050 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,050 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:03,050 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:54:03,062 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/9043ffee61f14698a7706ee664869392 2023-02-08 02:54:03,063 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,073 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,073 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,073 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,073 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,074 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,084 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/9db7edc503704ec4b15c6a01f16e7963 2023-02-08 02:54:03,090 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/9043ffee61f14698a7706ee664869392 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 2023-02-08 02:54:03,095 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:03,096 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/9db7edc503704ec4b15c6a01f16e7963 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 2023-02-08 02:54:03,101 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:03,102 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 52ms, sequenceid=6, compaction requested=false 2023-02-08 02:54:03,102 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-08 02:54:03,103 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:03,103 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:03,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:03,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,115 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,115 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,116 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,116 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:03,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e761e6f[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:03,118 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:03,118 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:03,130 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,131 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:03,131 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:03,140 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,140 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,140 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,140 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,141 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,142 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,151 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,151 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,151 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:03,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:03,155 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:03,155 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:03,155 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:03,155 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:03,155 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,169 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,169 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,169 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,171 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,171 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,171 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,183 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,183 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,183 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:03,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:03,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,183 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,183 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:03,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,184 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,285 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:03,285 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,285 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:03,307 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,308 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:03,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:03,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:03,311 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:03,311 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,312 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:03,312 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:03,312 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,312 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,319 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,320 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,320 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,321 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,321 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,321 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,321 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,322 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,322 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,321 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,323 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,323 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,324 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:03,324 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,324 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:03,334 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/5bfba10399d04290b63618221e785124 2023-02-08 02:54:03,336 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,347 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,347 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,347 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,347 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,347 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,352 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/95d4dff61c834774a13d12d3b5c106f3 2023-02-08 02:54:03,357 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/5bfba10399d04290b63618221e785124 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 2023-02-08 02:54:03,363 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:03,364 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/95d4dff61c834774a13d12d3b5c106f3 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 2023-02-08 02:54:03,368 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:03,370 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 45ms, sequenceid=19, compaction requested=false 2023-02-08 02:54:03,370 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:03,370 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,370 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:03,370 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:03,370 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,370 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,370 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,382 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,383 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,383 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:03,385 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58f8b9af[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:03,385 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:03,385 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,393 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,393 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:03,393 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:03,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:03,394 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:03,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,452 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,452 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,452 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,452 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,452 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,452 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,454 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:03,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:03,457 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:03,458 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:03,458 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:03,458 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:03,458 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,467 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,467 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,468 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,468 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,470 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,470 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,470 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:03,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:03,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,484 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,485 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:03,485 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:03,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,586 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:03,586 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,586 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:03,609 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,609 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:03,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:03,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:03,612 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:03,612 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,613 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:03,613 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:03,613 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,613 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,625 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,625 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,625 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,625 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,627 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,627 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,627 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,627 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,627 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,627 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:03,628 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,628 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,629 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:03,629 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,629 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,629 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:03,629 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,635 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,642 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/965b903623f44d5291ffd98c3f8513ff 2023-02-08 02:54:03,660 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/6c3fcb2ec2a64395bebb008c2f14781a 2023-02-08 02:54:03,666 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/965b903623f44d5291ffd98c3f8513ff as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff 2023-02-08 02:54:03,672 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:03,673 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/6c3fcb2ec2a64395bebb008c2f14781a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a 2023-02-08 02:54:03,678 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:03,679 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 50ms, sequenceid=32, compaction requested=true 2023-02-08 02:54:03,679 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:03,679 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,679 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:03,679 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:03,680 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,680 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,680 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,688 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:03,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ac325d4[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:03,690 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:03,690 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,698 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,699 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:03,699 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:03,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,709 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,709 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,719 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,719 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,719 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:03,720 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,720 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,720 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:03,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:03,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:03,724 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:03,724 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:03,724 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:03,724 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:03,724 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,736 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,737 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,738 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,739 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,739 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,751 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:03,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:03,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:03,752 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,752 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:03,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:03,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,853 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:03,853 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,853 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:03,879 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:03,879 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:03,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:03,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:03,885 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:03,885 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,885 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:03,885 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:03,886 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,886 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,930 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,930 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,930 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,930 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:03,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:03,931 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,931 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:03,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,932 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:03,932 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,932 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,932 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,932 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:03,933 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,932 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,933 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,933 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:03,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,934 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,936 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,936 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,936 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:03,936 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:03,936 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:03,936 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,936 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:03,937 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:03,951 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/d1ac453cff074c5d990081c955580b3e 2023-02-08 02:54:03,957 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,965 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/a75a806eb68249a5b59eb5d575ab4f40 2023-02-08 02:54:03,968 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,968 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,968 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,968 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,968 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,971 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/d1ac453cff074c5d990081c955580b3e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e 2023-02-08 02:54:03,976 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:03,977 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/a75a806eb68249a5b59eb5d575ab4f40 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 2023-02-08 02:54:03,982 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:03,983 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 46ms, sequenceid=45, compaction requested=true 2023-02-08 02:54:03,983 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:03,983 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:03,983 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:03,983 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:03,983 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,983 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:03,983 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:03,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,993 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:03,994 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:03,994 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:03,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:03,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@635e02ab[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:03,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:03,996 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,004 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,004 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:04,005 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:04,014 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,014 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,014 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,024 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,025 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,025 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,025 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,025 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,026 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:04,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:04,032 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:04,032 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:04,032 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:04,032 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:04,032 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,042 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,043 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,043 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,046 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,046 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,046 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:04,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:04,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,158 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:04,158 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:04,158 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:04,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:04,182 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:04,183 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:04,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:04,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:04,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:04,186 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:04,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:04,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:04,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,193 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,194 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:04,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,195 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,195 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,195 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,197 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,197 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,197 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,197 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,197 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,199 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,199 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:04,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,200 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,200 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,200 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,200 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:04,203 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,211 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/63a1f0af566846d68995d01dbbfb8745 2023-02-08 02:54:04,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,214 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,214 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,226 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/0b74ec66195544b89de22ed89340792d 2023-02-08 02:54:04,232 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/63a1f0af566846d68995d01dbbfb8745 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 2023-02-08 02:54:04,237 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:04,238 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/0b74ec66195544b89de22ed89340792d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d 2023-02-08 02:54:04,244 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:04,245 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 45ms, sequenceid=58, compaction requested=true 2023-02-08 02:54:04,245 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:04,245 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:04,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:04,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:04,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,256 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,256 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,256 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:04,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d35edc9[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:04,258 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:04,258 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,267 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,267 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:04,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,267 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:04,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,267 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,268 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,267 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,268 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,268 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:04,268 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:04,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,288 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:04,291 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:04,291 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:04,291 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:04,291 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:04,291 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,331 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,332 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,332 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,332 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,334 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,334 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,334 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:04,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:04,351 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:04,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,353 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,353 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,452 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:04,452 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:04,452 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:04,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:04,473 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:04,473 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:04,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:04,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:04,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-08 02:54:04,476 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-08 02:54:04,476 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,476 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-08 02:54:04,476 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-08 02:54:04,477 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,477 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,488 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,488 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,488 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-08 02:54:04,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-08 02:54:04,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-08 02:54:04,490 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,490 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. started... 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,490 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:04,498 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,503 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/25c33bdffb7c42478bfd1838821e482a 2023-02-08 02:54:04,509 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,509 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,509 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,509 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,509 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,519 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/32983f25ead4418f966a8e4781f58908 2023-02-08 02:54:04,525 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/25c33bdffb7c42478bfd1838821e482a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a 2023-02-08 02:54:04,532 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:04,533 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/32983f25ead4418f966a8e4781f58908 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 2023-02-08 02:54:04,539 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:04,540 INFO [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 50ms, sequenceid=71, compaction requested=true 2023-02-08 02:54:04,540 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for edae7e6dcabc9beb6fca230c3a7df74c: 2023-02-08 02:54:04,540 DEBUG [rs(jenkins-hbase12.apache.org,36075,1675824833838)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:04,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:04,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:04,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-08 02:54:04,541 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-08 02:54:04,551 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,551 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,551 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,551 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:04,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-08 02:54:04,553 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@101d0801[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:04,553 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-08 02:54:04,553 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,562 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:04,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:04,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-08 02:54:04,562 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:04,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,572 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,582 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-08 02:54:04,583 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,583 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-08 02:54:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase12.apache.org,36075,1675824833838': 2023-02-08 02:54:04,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-08 02:54:04,585 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-08 02:54:04,585 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:04,585 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:04,585 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-08 02:54:04,585 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,593 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,594 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:04,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,594 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:04,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,594 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,596 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,596 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:04,596 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-08 02:54:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,604 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:04,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:04,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-08 02:54:04,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:04,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:04,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,604 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-08 02:54:04,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-08 02:54:04,704 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:04,705 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:04,705 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:04,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:04,970 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 02:54:04,971 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:04,973 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:04,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:04,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:04,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:04,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:04,975 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:04,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:04,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:05,223 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,224 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:05,224 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:05,224 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:05,226 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-08 02:54:05,227 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:05,227 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:05,227 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:05,227 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:05,228 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:05,232 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 02:54:05,234 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:05,235 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:05,236 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:05,236 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:05,236 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,237 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:05,237 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,238 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,238 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:05,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:05,239 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,239 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,239 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,240 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-08 02:54:05,241 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:05,241 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:05,242 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824875242 2023-02-08 02:54:05,243 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,243 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:05,248 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:05,248 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:05,249 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:05,249 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824905249 2023-02-08 02:54:05,249 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824965249 2023-02-08 02:54:05,249 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 02:54:05,249 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:05,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:05,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:05,251 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:05,499 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,499 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:05,499 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:05,499 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:05,499 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:05,500 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:05,500 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:05,500 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:05,500 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:05,501 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:05,502 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 02:54:05,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:05,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:05,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:05,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:05,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:05,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:05,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:05,505 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,505 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,506 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,506 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 02:54:05,506 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:05,507 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:05,512 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824875512 2023-02-08 02:54:05,514 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,514 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:05,517 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:05,517 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:05,518 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:05,518 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824905518 2023-02-08 02:54:05,518 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824965518 2023-02-08 02:54:05,518 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:05,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:05,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN}] 2023-02-08 02:54:05,520 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:05,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:05,520 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:05,768 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,768 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:05,768 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:05,768 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:05,769 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:05,769 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:05,769 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:05,769 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:05,769 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:05,770 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:05,772 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 02:54:05,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:05,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:05,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:05,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:05,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:05,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:05,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:05,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:05,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,777 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,777 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:05,777 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:05,778 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:05,778 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824875778 2023-02-08 02:54:05,780 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:05,780 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:05,783 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:05,783 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:05,783 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:05,783 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824905783 2023-02-08 02:54:05,783 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824965783 2023-02-08 02:54:05,783 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:05,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN}] 2023-02-08 02:54:05,784 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:05,786 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:05,787 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:05,787 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824845787"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824845787"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824845787"}]},"ts":"1675824845787"} 2023-02-08 02:54:05,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:05,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:05,789 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:05,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:05,789 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:06,033 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,034 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:06,034 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:06,034 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:06,035 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:06,035 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:06,035 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:06,035 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:06,035 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:06,036 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:06,038 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:06,038 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:06,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:06,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:06,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:06,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:06,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:06,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:06,041 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,041 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,041 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:06,041 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,041 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,041 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,041 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:06,041 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,041 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:06,042 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:06,043 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:06,043 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c to restore RIT 2023-02-08 02:54:06,043 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824876043 2023-02-08 02:54:06,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,044 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:06,047 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:06,047 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:06,047 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:06,047 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824906047 2023-02-08 02:54:06,047 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824966047 2023-02-08 02:54:06,047 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:06,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:06,048 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824846047"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824846047"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824846047"}]},"ts":"1675824846047"} 2023-02-08 02:54:06,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:06,050 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:06,164 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:06,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:06,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing edae7e6dcabc9beb6fca230c3a7df74c, disabling compactions & flushes 2023-02-08 02:54:06,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:06,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:06,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. after waiting 0 ms 2023-02-08 02:54:06,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:06,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing edae7e6dcabc9beb6fca230c3a7df74c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:54:06,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/8ff8dd404de84fe5bf5cb22183cc7fde 2023-02-08 02:54:06,637 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/8c5d74b265d74aacaa8cf19d772ad624 2023-02-08 02:54:06,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf1/8ff8dd404de84fe5bf5cb22183cc7fde as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde 2023-02-08 02:54:06,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:54:06,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/.tmp/cf2/8c5d74b265d74aacaa8cf19d772ad624 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 2023-02-08 02:54:06,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:54:06,656 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 452ms, sequenceid=82, compaction requested=true 2023-02-08 02:54:06,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:54:06,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:06,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. 2023-02-08 02:54:06,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for edae7e6dcabc9beb6fca230c3a7df74c: Waiting for close lock at 1675824846204Running coprocessor pre-close hooks at 1675824846204Disabling compacts and flushes for region at 1675824846204Disabling writes for close at 1675824846204Obtaining lock to block concurrent updates at 1675824846204Preparing flush snapshotting stores in edae7e6dcabc9beb6fca230c3a7df74c at 1675824846204Finished memstore snapshotting testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824846204Flushing stores of testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. at 1675824846205 (+1 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf1: creating writer at 1675824846206 (+1 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf1: appending metadata at 1675824846209 (+3 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf1: closing flushed file at 1675824846209Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf2: creating writer at 1675824846628 (+419 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf2: appending metadata at 1675824846630 (+2 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf2: closing flushed file at 1675824846630Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf1: reopening flushed file at 1675824846643 (+13 ms)Flushing edae7e6dcabc9beb6fca230c3a7df74c/cf2: reopening flushed file at 1675824846650 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for edae7e6dcabc9beb6fca230c3a7df74c in 452ms, sequenceid=82, compaction requested=true at 1675824846656 (+6 ms)Writing region close event to WAL at 1675824846669 (+13 ms)Running coprocessor post-close hooks at 1675824846672 (+3 ms)Closed at 1675824846673 (+1 ms) 2023-02-08 02:54:06,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:06,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED 2023-02-08 02:54:06,677 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824846676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824846676"}]},"ts":"1675824846676"} 2023-02-08 02:54:06,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:06,678 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:06,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:06,678 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:06,798 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:06,798 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:06,799 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:06,800 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:06,801 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 02:54:06,801 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 02:54:06,801 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:06,801 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:06,801 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:06,802 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:06,802 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,802 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:06,802 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,803 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:06,803 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:06,803 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,804 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,804 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:06,804 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:06,804 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,804 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,804 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:06,804 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:06,804 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,804 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:06,805 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:06,805 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:06,806 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:06,806 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c to restore RIT 2023-02-08 02:54:06,806 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824876806 2023-02-08 02:54:06,807 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:06,807 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:06,809 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:06,809 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:06,809 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:06,809 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824906809 2023-02-08 02:54:06,809 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824966809 2023-02-08 02:54:06,809 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:06,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:06,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824846810"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824846810"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824846810"}]},"ts":"1675824846810"} 2023-02-08 02:54:06,812 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:06,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-08 02:54:06,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 in 762 msec 2023-02-08 02:54:06,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:06,815 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:06,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:06,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:07,060 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:07,060 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:07,061 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:07,061 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:07,061 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:07,063 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:07,063 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 02:54:07,063 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:07,063 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:07,064 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:07,064 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:07,064 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:07,064 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,065 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:07,065 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,065 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,065 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:07,066 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,066 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:07,066 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:07,066 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:07,066 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,066 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,066 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,066 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:07,067 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:07,067 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:07,068 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:07,068 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:07,068 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c to restore RIT 2023-02-08 02:54:07,068 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824877068 2023-02-08 02:54:07,069 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:07,069 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:07,072 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:07,072 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:07,072 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:07,073 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824907073 2023-02-08 02:54:07,073 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824967073 2023-02-08 02:54:07,073 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:07,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:07,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:07,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-08 02:54:07,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN in 1.2910 sec 2023-02-08 02:54:07,083 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:07,083 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:07,084 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:07,323 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,323 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:07,323 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:07,323 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:07,324 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:07,324 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:07,324 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:07,324 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:07,324 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:07,325 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:07,326 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:07,326 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:07,327 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:07,327 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:07,327 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:07,327 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:07,327 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:07,328 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,328 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:07,328 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,328 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:07,329 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:07,329 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,329 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:07,329 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:07,330 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:07,330 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:07,330 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:07,330 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 02:54:07,331 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:07,331 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:07,332 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824877332 2023-02-08 02:54:07,333 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:07,333 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,336 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:07,336 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:07,336 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:07,336 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824907336 2023-02-08 02:54:07,336 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824967336 2023-02-08 02:54:07,336 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:07,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,336 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:07,341 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:07,351 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=edae7e6dcabc9beb6fca230c3a7df74c, threads=14 2023-02-08 02:54:07,352 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,352 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,352 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,352 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,353 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,353 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,353 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,353 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,357 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,357 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,360 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,360 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,361 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,361 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,501 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,519 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,537 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,541 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,577 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,581 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,582 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,583 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,584 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,585 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,587 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,618 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,904 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,964 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:07,964 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region edae7e6dcabc9beb6fca230c3a7df74c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:08,000 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:08,000 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:08,000 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:08,087 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:08,087 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:08,088 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:08,089 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 02:54:08,089 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:08,089 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:08,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:08,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:08,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:08,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:08,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:08,092 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,092 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,092 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:08,093 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,093 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:08,093 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:08,093 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:08,093 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,093 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,093 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,093 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,094 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,094 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:08,094 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:08,095 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:08,095 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824878095 2023-02-08 02:54:08,096 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:08,096 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,099 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:08,099 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:08,099 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:08,099 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824908099 2023-02-08 02:54:08,099 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824968099 2023-02-08 02:54:08,099 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:08,109 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=edae7e6dcabc9beb6fca230c3a7df74c, threads=14 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,110 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,111 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,254 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,270 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,296 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,301 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,329 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,335 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,341 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,349 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,349 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,350 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,353 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,353 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,354 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,355 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde for region: edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:08,355 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region edae7e6dcabc9beb6fca230c3a7df74c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:08,395 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:08,401 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:08,403 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:08,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:08,403 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:08,403 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:08,599 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,599 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:08,600 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:08,600 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:08,601 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:08,602 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 02:54:08,602 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:08,603 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:08,603 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:08,603 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:08,603 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:08,604 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:08,604 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,604 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:08,605 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,605 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,605 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:08,606 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,606 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,606 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,606 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,606 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:08,607 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:08,608 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:08,608 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824878608 2023-02-08 02:54:08,609 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:08,609 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,611 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:08,611 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:08,611 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:08,612 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824908611 2023-02-08 02:54:08,612 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824968612 2023-02-08 02:54:08,612 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:08,615 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:08,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:08,617 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:08,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:08,862 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:08,862 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:08,862 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:08,862 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:08,862 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:08,862 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:08,862 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:08,863 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:08,863 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:08,863 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:08,864 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 02:54:08,864 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:08,864 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:08,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:08,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:08,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:08,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:08,866 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,866 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:08,866 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,866 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:08,866 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:08,867 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,867 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,867 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=edae7e6dcabc9beb6fca230c3a7df74c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,867 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure edae7e6dcabc9beb6fca230c3a7df74c, server=jenkins-hbase12.apache.org,36075,1675824833838 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:08,868 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 02:54:08,868 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:08,869 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:08,869 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824878869 2023-02-08 02:54:08,871 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:08,871 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:08,873 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:08,873 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=edae7e6dcabc9beb6fca230c3a7df74c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,36075,1675824833838, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838, openSeqNum=2 2023-02-08 02:54:08,873 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:08,873 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824908873 2023-02-08 02:54:08,873 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824968873 2023-02-08 02:54:08,873 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:08,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824848878"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675824848878"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675824848878"}]},"ts":"1675824848878"} 2023-02-08 02:54:08,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824848878"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824848878"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824848878"}]},"ts":"1675824848878"} 2023-02-08 02:54:08,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824848878"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824848878"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824848878"}]},"ts":"1675824848878"} 2023-02-08 02:54:08,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, ASSIGN}] 2023-02-08 02:54:08,889 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, ASSIGN 2023-02-08 02:54:08,891 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:54:08,892 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, ASSIGN 2023-02-08 02:54:08,892 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,36075,1675824833838; forceNewPlan=false, retain=false 2023-02-08 02:54:09,041 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:54:09,043 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=21c6f7f548611c6f9d9be71c188e4c15, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,043 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849043"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824849043"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824849043"}]},"ts":"1675824849043"} 2023-02-08 02:54:09,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 21c6f7f548611c6f9d9be71c188e4c15, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:09,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4be5e4800c36e1a14f75c459c9ec0f6a, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849048"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824849048"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824849048"}]},"ts":"1675824849048"} 2023-02-08 02:54:09,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 4be5e4800c36e1a14f75c459c9ec0f6a, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:09,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 21c6f7f548611c6f9d9be71c188e4c15, NAME => 'testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 02:54:09,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:09,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:09,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,203 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,204 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1 2023-02-08 02:54:09,204 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1 2023-02-08 02:54:09,205 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21c6f7f548611c6f9d9be71c188e4c15 columnFamilyName cf1 2023-02-08 02:54:09,226 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-top 2023-02-08 02:54:09,233 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-top 2023-02-08 02:54:09,240 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-top 2023-02-08 02:54:09,246 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-top 2023-02-08 02:54:09,251 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-top 2023-02-08 02:54:09,258 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-top 2023-02-08 02:54:09,263 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-top 2023-02-08 02:54:09,263 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(310): Store=21c6f7f548611c6f9d9be71c188e4c15/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:09,263 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,264 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2 2023-02-08 02:54:09,264 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2 2023-02-08 02:54:09,265 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21c6f7f548611c6f9d9be71c188e4c15 columnFamilyName cf2 2023-02-08 02:54:09,283 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-top 2023-02-08 02:54:09,289 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-top 2023-02-08 02:54:09,294 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-top 2023-02-08 02:54:09,300 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-top 2023-02-08 02:54:09,307 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-top 2023-02-08 02:54:09,312 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-top 2023-02-08 02:54:09,318 DEBUG [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-top 2023-02-08 02:54:09,318 INFO [StoreOpener-21c6f7f548611c6f9d9be71c188e4c15-1] regionserver.HStore(310): Store=21c6f7f548611c6f9d9be71c188e4c15/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:09,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:09,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:09,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 21c6f7f548611c6f9d9be71c188e4c15; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60606417, jitterRate=-0.09689401090145111}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:09,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 21c6f7f548611c6f9d9be71c188e4c15: Running coprocessor pre-open hook at 1675824849202Writing region info on filesystem at 1675824849202Initializing all the Stores at 1675824849203 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824849203Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824849203Cleaning up temporary data from old regions at 1675824849321 (+118 ms)Cleaning up detritus from prior splits at 1675824849322 (+1 ms)Running coprocessor post-open hooks at 1675824849324 (+2 ms)Region opened successfully at 1675824849325 (+1 ms) 2023-02-08 02:54:09,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15., pid=33, masterSystemTime=1675824849197 2023-02-08 02:54:09,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:09,368 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:09,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:09,368 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:09,368 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,368 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,368 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1912): 21c6f7f548611c6f9d9be71c188e4c15/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:09,368 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1912): 21c6f7f548611c6f9d9be71c188e4c15/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:09,368 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 21c6f7f548611c6f9d9be71c188e4c15/cf1 in testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,368 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 21c6f7f548611c6f9d9be71c188e4c15/cf2 in testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,369 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/.tmp, totalSize=34.9 K 2023-02-08 02:54:09,369 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/.tmp, totalSize=34.9 K 2023-02-08 02:54:09,370 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824843022 2023-02-08 02:54:09,370 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824843022 2023-02-08 02:54:09,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:09,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,370 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824843287 2023-02-08 02:54:09,370 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824843287 2023-02-08 02:54:09,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4be5e4800c36e1a14f75c459c9ec0f6a, NAME => 'testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 02:54:09,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:09,371 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824843588 2023-02-08 02:54:09,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,371 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824843588 2023-02-08 02:54:09,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:09,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=21c6f7f548611c6f9d9be71c188e4c15, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849372"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824849372"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824849372"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824849372"}]},"ts":"1675824849372"} 2023-02-08 02:54:09,372 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824843856 2023-02-08 02:54:09,372 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824843856 2023-02-08 02:54:09,373 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824844161 2023-02-08 02:54:09,373 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824844161 2023-02-08 02:54:09,374 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824844454 2023-02-08 02:54:09,374 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,374 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824844454 2023-02-08 02:54:09,375 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824844707 2023-02-08 02:54:09,375 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824844707 2023-02-08 02:54:09,376 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1 2023-02-08 02:54:09,377 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1 2023-02-08 02:54:09,377 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4be5e4800c36e1a14f75c459c9ec0f6a columnFamilyName cf1 2023-02-08 02:54:09,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-08 02:54:09,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 21c6f7f548611c6f9d9be71c188e4c15, server=jenkins-hbase12.apache.org,36075,1675824833838 in 329 msec 2023-02-08 02:54:09,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, ASSIGN in 495 msec 2023-02-08 02:54:09,392 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] throttle.PressureAwareThroughputController(145): 21c6f7f548611c6f9d9be71c188e4c15#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:09,398 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] throttle.PressureAwareThroughputController(145): 21c6f7f548611c6f9d9be71c188e4c15#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:09,402 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-bottom 2023-02-08 02:54:09,408 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-bottom 2023-02-08 02:54:09,412 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-bottom 2023-02-08 02:54:09,421 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-bottom 2023-02-08 02:54:09,426 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-bottom 2023-02-08 02:54:09,427 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/.tmp/cf2/29fe75ebd5d14d62bf6731b12ead8d73 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/29fe75ebd5d14d62bf6731b12ead8d73 2023-02-08 02:54:09,430 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-bottom 2023-02-08 02:54:09,434 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-bottom 2023-02-08 02:54:09,434 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 21c6f7f548611c6f9d9be71c188e4c15/cf2 of 21c6f7f548611c6f9d9be71c188e4c15 into 29fe75ebd5d14d62bf6731b12ead8d73(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:09,434 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 21c6f7f548611c6f9d9be71c188e4c15: 2023-02-08 02:54:09,434 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(310): Store=4be5e4800c36e1a14f75c459c9ec0f6a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:09,434 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15., storeName=21c6f7f548611c6f9d9be71c188e4c15/cf2, priority=9, startTime=1675824849368; duration=0sec 2023-02-08 02:54:09,434 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,434 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:09,435 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2 2023-02-08 02:54:09,435 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2 2023-02-08 02:54:09,435 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4be5e4800c36e1a14f75c459c9ec0f6a columnFamilyName cf2 2023-02-08 02:54:09,454 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-bottom 2023-02-08 02:54:09,459 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-bottom 2023-02-08 02:54:09,464 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-bottom 2023-02-08 02:54:09,470 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-bottom 2023-02-08 02:54:09,474 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-bottom 2023-02-08 02:54:09,478 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-bottom 2023-02-08 02:54:09,490 DEBUG [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-bottom 2023-02-08 02:54:09,490 INFO [StoreOpener-4be5e4800c36e1a14f75c459c9ec0f6a-1] regionserver.HStore(310): Store=4be5e4800c36e1a14f75c459c9ec0f6a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:09,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:09,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:09,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 4be5e4800c36e1a14f75c459c9ec0f6a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61501118, jitterRate=-0.08356192708015442}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:09,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 4be5e4800c36e1a14f75c459c9ec0f6a: Running coprocessor pre-open hook at 1675824849372Writing region info on filesystem at 1675824849372Initializing all the Stores at 1675824849373 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824849373Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824849373Cleaning up temporary data from old regions at 1675824849492 (+119 ms)Cleaning up detritus from prior splits at 1675824849493 (+1 ms)Running coprocessor post-open hooks at 1675824849495 (+2 ms)Region opened successfully at 1675824849495 2023-02-08 02:54:09,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a., pid=34, masterSystemTime=1675824849197 2023-02-08 02:54:09,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:09,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:09,496 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:09,497 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,497 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1912): 4be5e4800c36e1a14f75c459c9ec0f6a/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:09,497 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4be5e4800c36e1a14f75c459c9ec0f6a/cf1 in testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,497 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/.tmp, totalSize=34.9 K 2023-02-08 02:54:09,498 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824843022 2023-02-08 02:54:09,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,498 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824843287 2023-02-08 02:54:09,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=4be5e4800c36e1a14f75c459c9ec0f6a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,499 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824843588 2023-02-08 02:54:09,499 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849499"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824849499"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824849499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824849499"}]},"ts":"1675824849499"} 2023-02-08 02:54:09,499 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824843856 2023-02-08 02:54:09,500 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824844161 2023-02-08 02:54:09,500 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824844454 2023-02-08 02:54:09,501 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] compactions.Compactor(207): Compacting 8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824844707 2023-02-08 02:54:09,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-08 02:54:09,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 4be5e4800c36e1a14f75c459c9ec0f6a, server=jenkins-hbase12.apache.org,36075,1675824833838 in 450 msec 2023-02-08 02:54:09,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-08 02:54:09,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, ASSIGN in 620 msec 2023-02-08 02:54:09,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 in 4.5340 sec 2023-02-08 02:54:09,514 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] throttle.PressureAwareThroughputController(145): 4be5e4800c36e1a14f75c459c9ec0f6a#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:09,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36075] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:36774 deadline: 1675824909625, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c. is not online on jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,735 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:54:09,743 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 02:54:09,746 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:54:09,800 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:09,800 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:09,800 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:09,801 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:09,801 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:09,802 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-08 02:54:09,802 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitWithoutPONR 2023-02-08 02:54:09,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:09,805 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:09,806 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:09,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 02:54:09,807 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824849807"}]},"ts":"1675824849807"} 2023-02-08 02:54:09,808 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-08 02:54:09,821 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/.tmp/cf1/7fc641819ed646b28f2fa662fa286732 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/7fc641819ed646b28f2fa662fa286732 2023-02-08 02:54:09,828 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 21c6f7f548611c6f9d9be71c188e4c15/cf1 of 21c6f7f548611c6f9d9be71c188e4c15 into 7fc641819ed646b28f2fa662fa286732(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:09,828 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 21c6f7f548611c6f9d9be71c188e4c15: 2023-02-08 02:54:09,828 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15., storeName=21c6f7f548611c6f9d9be71c188e4c15/cf1, priority=9, startTime=1675824849367; duration=0sec 2023-02-08 02:54:09,828 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:09,828 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:09,828 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,828 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1912): 4be5e4800c36e1a14f75c459c9ec0f6a/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:09,828 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4be5e4800c36e1a14f75c459c9ec0f6a/cf2 in testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:09,829 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/.tmp, totalSize=34.9 K 2023-02-08 02:54:09,830 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824843022 2023-02-08 02:54:09,830 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824843287 2023-02-08 02:54:09,831 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824843588 2023-02-08 02:54:09,831 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824843856 2023-02-08 02:54:09,831 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824844161 2023-02-08 02:54:09,832 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824844454 2023-02-08 02:54:09,832 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] compactions.Compactor(207): Compacting 8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824844707 2023-02-08 02:54:09,843 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4be5e4800c36e1a14f75c459c9ec0f6a#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:09,856 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-08 02:54:09,856 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/.tmp/cf2/8126ffb42ecd40c4ab4ac24863ffca1a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8126ffb42ecd40c4ab4ac24863ffca1a 2023-02-08 02:54:09,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, UNASSIGN}] 2023-02-08 02:54:09,858 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, UNASSIGN 2023-02-08 02:54:09,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=21c6f7f548611c6f9d9be71c188e4c15, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,860 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849859"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824849859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824849859"}]},"ts":"1675824849859"} 2023-02-08 02:54:09,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 21c6f7f548611c6f9d9be71c188e4c15, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:09,863 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4be5e4800c36e1a14f75c459c9ec0f6a/cf2 of 4be5e4800c36e1a14f75c459c9ec0f6a into 8126ffb42ecd40c4ab4ac24863ffca1a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:09,863 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4be5e4800c36e1a14f75c459c9ec0f6a: 2023-02-08 02:54:09,864 INFO [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a., storeName=4be5e4800c36e1a14f75c459c9ec0f6a/cf2, priority=9, startTime=1675824849496; duration=0sec 2023-02-08 02:54:09,864 DEBUG [RS:1;jenkins-hbase12:36075-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:09,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, UNASSIGN 2023-02-08 02:54:09,866 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4be5e4800c36e1a14f75c459c9ec0f6a, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:09,866 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824849866"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824849866"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824849866"}]},"ts":"1675824849866"} 2023-02-08 02:54:09,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 4be5e4800c36e1a14f75c459c9ec0f6a, server=jenkins-hbase12.apache.org,36075,1675824833838}] 2023-02-08 02:54:09,907 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:09,907 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:09,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 02:54:09,930 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/.tmp/cf1/ac27dd75fdcd4f308a740a35772127fd as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/ac27dd75fdcd4f308a740a35772127fd 2023-02-08 02:54:09,936 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4be5e4800c36e1a14f75c459c9ec0f6a/cf1 of 4be5e4800c36e1a14f75c459c9ec0f6a into ac27dd75fdcd4f308a740a35772127fd(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:09,937 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4be5e4800c36e1a14f75c459c9ec0f6a: 2023-02-08 02:54:09,937 INFO [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a., storeName=4be5e4800c36e1a14f75c459c9ec0f6a/cf1, priority=9, startTime=1675824849496; duration=0sec 2023-02-08 02:54:09,937 DEBUG [RS:1;jenkins-hbase12:36075-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:10,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:10,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 4be5e4800c36e1a14f75c459c9ec0f6a, disabling compactions & flushes 2023-02-08 02:54:10,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:10,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:10,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. after waiting 0 ms 2023-02-08 02:54:10,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:10,025 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-bottom] to archive 2023-02-08 02:54:10,027 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:10,028 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,030 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,031 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,033 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,034 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,035 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,037 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,038 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-bottom] to archive 2023-02-08 02:54:10,039 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:10,040 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,042 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,043 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,044 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,045 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,047 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,048 DEBUG [StoreCloser-testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:54:10,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:10,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a. 2023-02-08 02:54:10,052 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 4be5e4800c36e1a14f75c459c9ec0f6a: Waiting for close lock at 1675824850015Running coprocessor pre-close hooks at 1675824850015Disabling compacts and flushes for region at 1675824850015Disabling writes for close at 1675824850015Writing region close event to WAL at 1675824850049 (+34 ms)Running coprocessor post-close hooks at 1675824850052 (+3 ms)Closed at 1675824850052 2023-02-08 02:54:10,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:10,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:10,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 21c6f7f548611c6f9d9be71c188e4c15, disabling compactions & flushes 2023-02-08 02:54:10,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:10,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:10,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. after waiting 0 ms 2023-02-08 02:54:10,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:10,055 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=4be5e4800c36e1a14f75c459c9ec0f6a, regionState=CLOSED 2023-02-08 02:54:10,055 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824850055"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824850055"}]},"ts":"1675824850055"} 2023-02-08 02:54:10,055 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde-top] to archive 2023-02-08 02:54:10,056 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:10,058 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-08 02:54:10,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 4be5e4800c36e1a14f75c459c9ec0f6a, server=jenkins-hbase12.apache.org,36075,1675824833838 in 189 msec 2023-02-08 02:54:10,059 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=4be5e4800c36e1a14f75c459c9ec0f6a, UNASSIGN in 202 msec 2023-02-08 02:54:10,061 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,062 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,064 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,065 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,066 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,068 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624-top] to archive 2023-02-08 02:54:10,069 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:10,070 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,071 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,072 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,074 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,075 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,076 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,077 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:54:10,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:10,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15. 2023-02-08 02:54:10,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 21c6f7f548611c6f9d9be71c188e4c15: Waiting for close lock at 1675824850054Running coprocessor pre-close hooks at 1675824850054Disabling compacts and flushes for region at 1675824850054Disabling writes for close at 1675824850054Writing region close event to WAL at 1675824850078 (+24 ms)Running coprocessor post-close hooks at 1675824850082 (+4 ms)Closed at 1675824850082 2023-02-08 02:54:10,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:10,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=21c6f7f548611c6f9d9be71c188e4c15, regionState=CLOSED 2023-02-08 02:54:10,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824850084"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824850084"}]},"ts":"1675824850084"} 2023-02-08 02:54:10,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-08 02:54:10,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 21c6f7f548611c6f9d9be71c188e4c15, server=jenkins-hbase12.apache.org,36075,1675824833838 in 225 msec 2023-02-08 02:54:10,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-08 02:54:10,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=21c6f7f548611c6f9d9be71c188e4c15, UNASSIGN in 231 msec 2023-02-08 02:54:10,091 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824850091"}]},"ts":"1675824850091"} 2023-02-08 02:54:10,093 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-08 02:54:10,104 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-08 02:54:10,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 304 msec 2023-02-08 02:54:10,108 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,108 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 02:54:10,110 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-08 02:54:10,110 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,110 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitWithoutPONR 2023-02-08 02:54:10,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,113 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,113 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,113 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,114 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 02:54:10,120 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:10,120 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:10,120 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,122 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/recovered.edits] 2023-02-08 02:54:10,123 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/recovered.edits] 2023-02-08 02:54:10,123 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/recovered.edits] 2023-02-08 02:54:10,126 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/.regioninfo to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/.regioninfo 2023-02-08 02:54:10,127 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/.regioninfo to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/.regioninfo 2023-02-08 02:54:10,128 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/25c33bdffb7c42478bfd1838821e482a 2023-02-08 02:54:10,130 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/5bfba10399d04290b63618221e785124 2023-02-08 02:54:10,131 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,132 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/63a1f0af566846d68995d01dbbfb8745 2023-02-08 02:54:10,132 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/25c33bdffb7c42478bfd1838821e482a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,133 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,134 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,134 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/8ff8dd404de84fe5bf5cb22183cc7fde 2023-02-08 02:54:10,134 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/5bfba10399d04290b63618221e785124.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,135 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/63a1f0af566846d68995d01dbbfb8745.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,135 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/9043ffee61f14698a7706ee664869392 2023-02-08 02:54:10,135 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,136 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/965b903623f44d5291ffd98c3f8513ff 2023-02-08 02:54:10,137 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/8ff8dd404de84fe5bf5cb22183cc7fde.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,137 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,138 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf1/d1ac453cff074c5d990081c955580b3e 2023-02-08 02:54:10,138 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/9043ffee61f14698a7706ee664869392.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,138 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,139 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/965b903623f44d5291ffd98c3f8513ff.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,140 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,141 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/d1ac453cff074c5d990081c955580b3e.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,141 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/0b74ec66195544b89de22ed89340792d 2023-02-08 02:54:10,143 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/32983f25ead4418f966a8e4781f58908 2023-02-08 02:54:10,143 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,144 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/0b74ec66195544b89de22ed89340792d.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,144 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,144 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/6c3fcb2ec2a64395bebb008c2f14781a 2023-02-08 02:54:10,146 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/32983f25ead4418f966a8e4781f58908.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,146 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,146 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/8c5d74b265d74aacaa8cf19d772ad624 2023-02-08 02:54:10,147 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/6c3fcb2ec2a64395bebb008c2f14781a.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,147 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,148 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/95d4dff61c834774a13d12d3b5c106f3 2023-02-08 02:54:10,149 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/9db7edc503704ec4b15c6a01f16e7963 2023-02-08 02:54:10,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8c5d74b265d74aacaa8cf19d772ad624.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,151 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,152 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/95d4dff61c834774a13d12d3b5c106f3.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,152 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/cf2/a75a806eb68249a5b59eb5d575ab4f40 2023-02-08 02:54:10,153 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/21c6f7f548611c6f9d9be71c188e4c15/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,153 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/9db7edc503704ec4b15c6a01f16e7963.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,155 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/a75a806eb68249a5b59eb5d575ab4f40.edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,155 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/7fc641819ed646b28f2fa662fa286732 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf1/7fc641819ed646b28f2fa662fa286732 2023-02-08 02:54:10,156 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c/recovered.edits/85.seqid 2023-02-08 02:54:10,156 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/edae7e6dcabc9beb6fca230c3a7df74c 2023-02-08 02:54:10,158 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/ac27dd75fdcd4f308a740a35772127fd to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf1/ac27dd75fdcd4f308a740a35772127fd 2023-02-08 02:54:10,158 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/29fe75ebd5d14d62bf6731b12ead8d73 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/cf2/29fe75ebd5d14d62bf6731b12ead8d73 2023-02-08 02:54:10,160 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8126ffb42ecd40c4ab4ac24863ffca1a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/cf2/8126ffb42ecd40c4ab4ac24863ffca1a 2023-02-08 02:54:10,161 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15/recovered.edits/90.seqid 2023-02-08 02:54:10,162 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:10,164 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a/recovered.edits/90.seqid 2023-02-08 02:54:10,165 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitWithoutPONR/4be5e4800c36e1a14f75c459c9ec0f6a 2023-02-08 02:54:10,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-08 02:54:10,168 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,171 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-08 02:54:10,174 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-08 02:54:10,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-08 02:54:10,175 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824850175"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:10,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824850175"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:10,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824850175"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:10,178 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:54:10,178 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => edae7e6dcabc9beb6fca230c3a7df74c, NAME => 'testSplitWithoutPONR,,1675824842516.edae7e6dcabc9beb6fca230c3a7df74c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4be5e4800c36e1a14f75c459c9ec0f6a, NAME => 'testSplitWithoutPONR,,1675824844971.4be5e4800c36e1a14f75c459c9ec0f6a.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 21c6f7f548611c6f9d9be71c188e4c15, NAME => 'testSplitWithoutPONR,41,1675824844971.21c6f7f548611c6f9d9be71c188e4c15.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 02:54:10,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-08 02:54:10,178 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824850178"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:10,180 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-08 02:54:10,193 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:10,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 84 msec 2023-02-08 02:54:10,214 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,214 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-08 02:54:10,215 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-08 02:54:10,228 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 395) Potentially hanging thread: hconnection-0x187a2033-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:51812 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53852 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:52052 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53826 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53656 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54166 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57328 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:46216 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:46632 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:46678 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:53724 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:52042 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:46774 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:57160 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:46492 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1000470655_15 at /127.0.0.1:54128 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=866 (was 837) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 259), ProcessCount=173 (was 173), AvailableMemoryMB=3680 (was 3814) 2023-02-08 02:54:10,243 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=866, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=173, AvailableMemoryMB=3679 2023-02-08 02:54:10,244 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:10,244 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,244 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:54:10,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:54:10,258 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:10,260 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:54:10,260 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:54:10,262 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,263 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 empty. 2023-02-08 02:54:10,263 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,263 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 02:54:10,274 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:10,275 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => a6003dd11a06ff730283e23c33e29876, NAME => 'testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:54:10,282 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:10,283 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing a6003dd11a06ff730283e23c33e29876, disabling compactions & flushes 2023-02-08 02:54:10,283 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,283 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,283 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. after waiting 0 ms 2023-02-08 02:54:10,283 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,283 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,283 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for a6003dd11a06ff730283e23c33e29876: Waiting for close lock at 1675824850283Disabling compacts and flushes for region at 1675824850283Disabling writes for close at 1675824850283Writing region close event to WAL at 1675824850283Closed at 1675824850283 2023-02-08 02:54:10,285 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:54:10,286 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824850285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824850285"}]},"ts":"1675824850285"} 2023-02-08 02:54:10,287 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:54:10,288 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:54:10,288 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824850288"}]},"ts":"1675824850288"} 2023-02-08 02:54:10,290 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 02:54:10,335 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:10,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:10,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:10,336 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:10,337 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:54:10,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN}] 2023-02-08 02:54:10,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:10,339 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:54:10,490 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:10,491 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,491 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824850491"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824850491"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824850491"}]},"ts":"1675824850491"} 2023-02-08 02:54:10,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:10,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6003dd11a06ff730283e23c33e29876, NAME => 'testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:10,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:10,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:10,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,654 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,656 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1 2023-02-08 02:54:10,656 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1 2023-02-08 02:54:10,656 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6003dd11a06ff730283e23c33e29876 columnFamilyName cf1 2023-02-08 02:54:10,657 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(310): Store=a6003dd11a06ff730283e23c33e29876/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:10,657 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,659 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2 2023-02-08 02:54:10,659 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2 2023-02-08 02:54:10,659 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6003dd11a06ff730283e23c33e29876 columnFamilyName cf2 2023-02-08 02:54:10,660 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(310): Store=a6003dd11a06ff730283e23c33e29876/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:10,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:10,665 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:10,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:54:10,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened a6003dd11a06ff730283e23c33e29876; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74027589, jitterRate=0.10309703648090363}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:10,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for a6003dd11a06ff730283e23c33e29876: Running coprocessor pre-open hook at 1675824850653Writing region info on filesystem at 1675824850653Initializing all the Stores at 1675824850654 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824850654Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824850654Cleaning up temporary data from old regions at 1675824850662 (+8 ms)Cleaning up detritus from prior splits at 1675824850663 (+1 ms)Running coprocessor post-open hooks at 1675824850667 (+4 ms)Region opened successfully at 1675824850667 2023-02-08 02:54:10,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876., pid=43, masterSystemTime=1675824850645 2023-02-08 02:54:10,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,670 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,670 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824850670"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824850670"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824850670"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824850670"}]},"ts":"1675824850670"} 2023-02-08 02:54:10,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-08 02:54:10,674 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 in 179 msec 2023-02-08 02:54:10,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-08 02:54:10,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN in 337 msec 2023-02-08 02:54:10,676 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:54:10,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824850676"}]},"ts":"1675824850676"} 2023-02-08 02:54:10,678 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 02:54:10,699 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:54:10,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 442 msec 2023-02-08 02:54:10,761 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 02:54:10,765 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:32972, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 02:54:10,773 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:10,773 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:10,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:10,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:10,776 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:10,776 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:10,776 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:10,776 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:10,776 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,776 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,789 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:10,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:10,793 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:10,793 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:10,793 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:10,793 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:10,793 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:10,793 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:10,795 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:54:10,803 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,810 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/b689ce85b52f4fe5b49d4f9b88e2305b 2023-02-08 02:54:10,815 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,815 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:10,815 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,815 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,815 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:10,828 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/dd8e19326dbc4573a001176d728944cb 2023-02-08 02:54:10,834 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/b689ce85b52f4fe5b49d4f9b88e2305b as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b 2023-02-08 02:54:10,840 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:10,842 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/dd8e19326dbc4573a001176d728944cb as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb 2023-02-08 02:54:10,847 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:10,848 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 53ms, sequenceid=6, compaction requested=false 2023-02-08 02:54:10,848 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-08 02:54:10,849 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:10,849 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:10,849 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:10,849 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:10,849 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,849 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:10,849 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:10,856 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,856 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:10,856 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,856 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:10,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:10,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:10,858 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:10,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23437253[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:10,858 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:10,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,866 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:10,867 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:10,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:10,867 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:10,877 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:10,877 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:10,877 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:10,887 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:10,887 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:10,887 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:10,887 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,888 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:10,888 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:10,888 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:10,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:10,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:10,891 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:10,891 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:10,891 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:10,891 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:10,891 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:10,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:10,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:10,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:10,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:10,900 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,901 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,901 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:10,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,905 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:10,906 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:10,906 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:10,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:10,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:10,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:10,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:10,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:10,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:10,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,921 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:10,921 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:10,921 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:10,921 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:10,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:10,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:10,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,021 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:11,021 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,021 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:11,041 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,041 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:11,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:11,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:11,044 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,044 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,044 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:11,044 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:11,044 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,044 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,057 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,057 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:11,058 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:11,058 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,058 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:11,059 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:11,066 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,066 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,066 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,066 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,067 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,066 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,072 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/b2bc62d9838445478da0cf9ec953a31e 2023-02-08 02:54:11,084 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/534653f171564dd8b8522cde21341881 2023-02-08 02:54:11,088 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/b2bc62d9838445478da0cf9ec953a31e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e 2023-02-08 02:54:11,092 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:11,093 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/534653f171564dd8b8522cde21341881 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 2023-02-08 02:54:11,097 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:11,098 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 39ms, sequenceid=19, compaction requested=false 2023-02-08 02:54:11,098 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:11,098 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,098 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:11,098 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:11,098 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,098 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,098 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,108 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,108 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:11,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@309b67ca[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:11,110 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:11,110 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,119 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,119 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:11,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,119 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,120 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,120 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:11,157 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,157 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,157 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,178 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,178 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,178 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,178 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,178 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,179 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,179 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:11,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:11,183 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:11,183 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:11,183 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:11,183 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:11,183 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:11,192 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,192 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,192 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,194 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,194 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:11,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:11,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,204 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,205 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,304 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:11,304 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,305 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:11,324 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,324 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:11,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:11,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:11,327 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,327 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,327 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:11,327 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:11,327 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,328 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,340 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,340 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,340 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:11,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,341 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,342 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:11,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:11,343 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,343 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:11,343 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:11,350 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,358 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/f3199f6bd66a4e309b8c65f65a54ee65 2023-02-08 02:54:11,361 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,361 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,361 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,361 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,361 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,373 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/8d725120ea84482abd7e3a4fbdbdf10f 2023-02-08 02:54:11,378 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/f3199f6bd66a4e309b8c65f65a54ee65 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 2023-02-08 02:54:11,383 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:11,384 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/8d725120ea84482abd7e3a4fbdbdf10f as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f 2023-02-08 02:54:11,389 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:11,390 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 47ms, sequenceid=32, compaction requested=true 2023-02-08 02:54:11,390 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:11,390 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,390 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:11,390 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:11,390 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,390 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,390 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,403 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,403 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,403 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,403 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:11,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12e28fb5[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:11,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:11,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,414 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:11,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:11,414 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:11,424 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,424 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,424 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,435 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,435 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:11,438 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:11,438 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:11,438 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:11,438 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:11,438 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:11,438 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,446 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,445 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,446 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,446 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,448 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,448 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,448 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:11,484 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,484 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,585 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:11,586 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,586 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:11,604 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,604 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:11,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:11,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:11,606 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,607 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,607 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:11,607 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:11,607 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,607 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,624 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,624 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,624 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,624 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,624 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,625 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:11,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,626 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,626 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:11,626 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,626 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,626 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:11,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,635 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,640 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/724a840295fe403eb58bd6fc1fcadf65 2023-02-08 02:54:11,652 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/82814f70319843b6a8229e958372e1d1 2023-02-08 02:54:11,657 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/724a840295fe403eb58bd6fc1fcadf65 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 2023-02-08 02:54:11,661 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:11,662 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/82814f70319843b6a8229e958372e1d1 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 2023-02-08 02:54:11,666 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:11,667 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 41ms, sequenceid=45, compaction requested=true 2023-02-08 02:54:11,667 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:11,668 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,668 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:11,668 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:11,668 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,668 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,668 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,677 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,677 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,677 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,677 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:11,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ac7c0e8[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:11,679 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:11,679 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,688 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,688 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:11,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:11,688 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,698 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,698 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:11,701 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:11,701 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:11,701 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:11,701 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:11,701 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,709 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,709 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,709 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,709 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:11,719 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:11,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:11,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,720 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,820 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:11,820 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,820 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:11,837 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:11,837 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:11,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:11,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:11,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:11,840 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:11,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:11,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:11,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,852 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,852 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:11,852 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:11,853 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,853 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:11,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,853 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:11,854 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,854 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,854 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:11,855 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:11,855 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:11,866 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/bb89df2970984436a5b2321f361082e5 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,866 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,878 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/b36a2620eced404599e093e53bb6cee0 2023-02-08 02:54:11,883 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/bb89df2970984436a5b2321f361082e5 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 2023-02-08 02:54:11,887 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:11,888 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/b36a2620eced404599e093e53bb6cee0 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 2023-02-08 02:54:11,892 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:11,893 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 38ms, sequenceid=58, compaction requested=true 2023-02-08 02:54:11,893 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:11,893 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:11,893 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:11,893 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:11,894 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,894 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:11,894 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:11,905 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,905 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:11,905 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,905 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:11,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:11,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:11,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:11,907 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:11,907 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@522ee609[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:11,907 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:11,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:11,919 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,919 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:11,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:11,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:11,980 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:11,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,000 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:11,999 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:12,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:12,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:12,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:12,003 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:12,003 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:12,003 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:12,003 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:12,003 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:12,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:12,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:12,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,015 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:12,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,016 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,016 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:12,016 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:12,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,024 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:12,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,125 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:12,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:12,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:12,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:12,141 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:12,141 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:12,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:12,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:12,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-08 02:54:12,143 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-08 02:54:12,144 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:12,144 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-08 02:54:12,144 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:54:12,144 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,144 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,150 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,151 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,151 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:12,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:12,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:12,153 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:12,153 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:12,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:12,153 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. started... 2023-02-08 02:54:12,153 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:12,161 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,161 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/38691f4209a44316b0db128e76fdabad 2023-02-08 02:54:12,171 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,171 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,171 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:12,172 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,172 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:12,174 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/49f14693a2314106ad8cc7cc565d8ddb 2023-02-08 02:54:12,174 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:12,182 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/38691f4209a44316b0db128e76fdabad as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad 2023-02-08 02:54:12,187 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:12,188 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/49f14693a2314106ad8cc7cc565d8ddb as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb 2023-02-08 02:54:12,194 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:12,195 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 42ms, sequenceid=71, compaction requested=true 2023-02-08 02:54:12,195 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:12,196 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:12,196 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:12,196 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:12,196 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,196 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-08 02:54:12,196 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-08 02:54:12,203 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,203 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:12,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:12,203 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:12,203 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:12,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-08 02:54:12,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39e06eb1[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:12,205 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:54:12,206 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,213 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:12,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:12,214 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,214 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:12,214 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:12,224 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,224 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:12,224 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,224 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:12,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:12,235 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:12,235 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,235 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:12,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-08 02:54:12,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-08 02:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:12,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:54:12,248 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-08 02:54:12,248 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:12,248 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:12,248 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-08 02:54:12,248 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,256 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,256 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,256 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:12,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,257 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,258 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,258 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,259 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-08 02:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-08 02:54:12,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,362 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:12,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:12,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,362 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:12,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-08 02:54:12,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-08 02:54:12,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:12,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:12,463 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:12,463 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:12,463 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:12,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:12,726 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 02:54:12,727 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:12,728 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,729 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:12,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:12,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:12,978 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,978 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:12,978 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:12,978 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:12,979 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-08 02:54:12,979 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:12,979 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:12,979 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:12,979 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:12,980 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:12,981 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 02:54:12,982 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:12,982 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:12,982 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:12,982 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:12,983 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:12,983 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:12,983 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:12,983 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:12,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:12,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:12,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:12,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:12,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:12,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:12,986 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,986 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,986 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,986 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:12,987 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:12,988 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:12,988 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824882988 2023-02-08 02:54:12,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,989 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:12,992 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:12,992 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:12,992 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:12,992 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824912992 2023-02-08 02:54:12,992 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824972992 2023-02-08 02:54:12,992 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:12,992 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:12,994 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:12,994 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:12,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:12,995 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:13,242 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:13,243 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:13,243 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:13,244 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:13,245 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 02:54:13,245 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:13,246 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:13,246 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:13,246 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:13,246 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:13,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:13,248 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,249 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,249 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,249 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 02:54:13,249 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:13,250 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:13,250 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824883250 2023-02-08 02:54:13,251 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,252 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:13,254 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,254 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,254 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:13,254 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824913254 2023-02-08 02:54:13,254 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824973254 2023-02-08 02:54:13,254 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:13,254 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:13,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN}] 2023-02-08 02:54:13,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:13,257 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:13,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:13,505 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:13,505 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:13,506 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:13,508 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 02:54:13,508 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:13,509 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:13,509 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:13,509 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:13,509 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,510 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:13,510 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,510 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,511 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,511 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:13,511 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,512 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,512 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:13,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:13,513 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,513 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,513 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,513 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:13,514 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:13,515 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:13,515 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824883515 2023-02-08 02:54:13,516 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:13,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,519 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,519 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,519 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:13,519 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824913519 2023-02-08 02:54:13,519 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824973519 2023-02-08 02:54:13,519 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:13,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN}] 2023-02-08 02:54:13,519 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:13,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:13,523 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:13,523 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824853523"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824853523"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824853523"}]},"ts":"1675824853523"} 2023-02-08 02:54:13,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:13,525 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:13,525 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:13,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:13,525 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:13,769 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:13,770 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:13,770 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:13,771 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:13,772 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:13,772 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:13,772 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:13,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:13,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:13,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:13,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,773 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:13,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:13,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,774 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:13,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:13,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:13,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:13,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:13,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:13,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:13,776 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:13,776 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:13,777 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:13,777 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:13,777 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:13,777 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824883777 2023-02-08 02:54:13,778 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:13,778 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:13,781 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,781 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSING, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:13,781 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:13,782 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824913782 2023-02-08 02:54:13,782 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824973782 2023-02-08 02:54:13,782 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:13,782 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:13,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824853782"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824853782"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824853782"}]},"ts":"1675824853782"} 2023-02-08 02:54:13,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:13,784 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:13,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:13,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing a6003dd11a06ff730283e23c33e29876, disabling compactions & flushes 2023-02-08 02:54:13,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:13,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:13,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. after waiting 0 ms 2023-02-08 02:54:13,938 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:13,938 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing a6003dd11a06ff730283e23c33e29876 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:54:13,952 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/d346a3302bb44494a1889ba354b94097 2023-02-08 02:54:13,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/6c138bb62d9c498bac4fca179071b865 2023-02-08 02:54:13,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/d346a3302bb44494a1889ba354b94097 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 2023-02-08 02:54:13,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:54:13,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/6c138bb62d9c498bac4fca179071b865 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 2023-02-08 02:54:13,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:54:13,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 40ms, sequenceid=82, compaction requested=true 2023-02-08 02:54:13,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:54:13,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:13,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:13,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for a6003dd11a06ff730283e23c33e29876: Waiting for close lock at 1675824853938Running coprocessor pre-close hooks at 1675824853938Disabling compacts and flushes for region at 1675824853938Disabling writes for close at 1675824853938Obtaining lock to block concurrent updates at 1675824853938Preparing flush snapshotting stores in a6003dd11a06ff730283e23c33e29876 at 1675824853938Finished memstore snapshotting testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824853938Flushing stores of testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. at 1675824853939 (+1 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf1: creating writer at 1675824853940 (+1 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf1: appending metadata at 1675824853942 (+2 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf1: closing flushed file at 1675824853942Flushing a6003dd11a06ff730283e23c33e29876/cf2: creating writer at 1675824853956 (+14 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf2: appending metadata at 1675824853958 (+2 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf2: closing flushed file at 1675824853958Flushing a6003dd11a06ff730283e23c33e29876/cf1: reopening flushed file at 1675824853969 (+11 ms)Flushing a6003dd11a06ff730283e23c33e29876/cf2: reopening flushed file at 1675824853974 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a6003dd11a06ff730283e23c33e29876 in 40ms, sequenceid=82, compaction requested=true at 1675824853978 (+4 ms)Writing region close event to WAL at 1675824853979 (+1 ms)Running coprocessor post-close hooks at 1675824853982 (+3 ms)Closed at 1675824853983 (+1 ms) 2023-02-08 02:54:13,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:13,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED 2023-02-08 02:54:13,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824853986"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824853986"}]},"ts":"1675824853986"} 2023-02-08 02:54:13,987 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:13,987 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:13,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:13,987 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:14,032 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,032 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:14,032 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:14,033 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:14,033 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:14,033 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:14,033 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:14,033 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:14,033 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:14,034 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:14,035 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:14,035 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 02:54:14,035 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-08 02:54:14,036 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:14,036 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:14,036 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:14,037 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:14,037 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,037 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:14,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:14,038 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,039 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:14,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:14,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:14,040 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,040 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,040 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:14,040 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,041 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:14,041 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:14,041 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:14,042 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:14,042 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:14,042 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824884042 2023-02-08 02:54:14,043 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:14,043 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:14,046 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,046 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,046 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:14,046 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824914046 2023-02-08 02:54:14,047 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824974047 2023-02-08 02:54:14,047 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:14,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:14,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824854047"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824854047"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824854047"}]},"ts":"1675824854047"} 2023-02-08 02:54:14,049 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:14,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-08 02:54:14,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 in 265 msec 2023-02-08 02:54:14,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:14,052 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:14,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:14,297 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,297 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:14,297 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:14,297 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:14,298 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:14,298 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:14,298 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:14,298 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:14,298 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:14,299 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:14,300 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:14,300 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-08 02:54:14,300 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:14,301 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:14,301 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:14,302 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:14,302 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:14,302 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,302 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:14,303 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,303 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,303 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,303 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:14,303 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,304 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,304 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:14,305 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,305 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,305 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:14,305 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:14,305 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,306 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,306 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,306 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:14,306 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:14,306 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:14,307 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:14,307 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:14,307 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:14,308 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824884308 2023-02-08 02:54:14,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,309 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:14,311 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,311 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,311 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:14,311 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824914311 2023-02-08 02:54:14,311 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824974311 2023-02-08 02:54:14,311 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:14,312 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,313 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:14,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-08 02:54:14,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN in 794 msec 2023-02-08 02:54:14,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:14,319 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:14,319 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:14,562 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,562 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:14,562 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:14,562 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:14,562 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:14,562 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:14,563 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:14,563 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:14,563 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:14,564 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:14,565 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:14,565 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:14,565 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:14,565 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:14,566 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:14,566 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:14,566 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:14,566 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,567 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:14,567 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,567 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:14,567 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:14,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:14,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:14,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:14,570 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:14,570 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:14,571 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:14,571 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824884571 2023-02-08 02:54:14,572 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,572 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:14,575 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,575 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:14,575 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:14,575 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824914575 2023-02-08 02:54:14,575 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824974575 2023-02-08 02:54:14,575 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:14,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:14,589 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=a6003dd11a06ff730283e23c33e29876, threads=14 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,591 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,592 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,592 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,592 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,592 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,762 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,773 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,781 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,790 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,791 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,792 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,801 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,813 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,819 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,825 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,827 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,829 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,829 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,832 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:14,832 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region a6003dd11a06ff730283e23c33e29876 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:14,864 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:14,864 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:14,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:14,865 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:15,076 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,076 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:15,076 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:15,076 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:15,078 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:15,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:15,079 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:15,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:15,079 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:15,080 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:15,081 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 02:54:15,081 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:15,081 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:15,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:15,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:15,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:15,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:15,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:15,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:15,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:15,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:15,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:15,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:15,086 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,086 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,086 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:15,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:15,086 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:15,087 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:15,088 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:15,088 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824885088 2023-02-08 02:54:15,089 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,089 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:15,091 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:15,092 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:15,092 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:15,092 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824915092 2023-02-08 02:54:15,092 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824975092 2023-02-08 02:54:15,092 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:15,100 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=a6003dd11a06ff730283e23c33e29876, threads=14 2023-02-08 02:54:15,101 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,105 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,105 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,105 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,105 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,105 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,106 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,106 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,108 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,108 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,109 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,113 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,113 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,113 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,297 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,307 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,308 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,308 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,308 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,315 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,319 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,337 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,337 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,345 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,349 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,350 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,352 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,655 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 for region: a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:15,655 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region a6003dd11a06ff730283e23c33e29876 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:15,690 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:15,696 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/20c6efed62991c27b5a3a43523a99902/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:15,698 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/553b6d198ab5781a750a6fe711553d74/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:54:15,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:15,698 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:15,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:15,842 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:15,843 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:15,843 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:15,844 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:15,844 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:15,846 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 02:54:15,846 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:15,846 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:15,846 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:15,847 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:15,847 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:15,847 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:15,848 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,848 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:15,848 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,848 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:15,849 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,849 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:15,850 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,850 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:15,850 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:15,851 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,851 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:15,851 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:15,853 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:15,853 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,853 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,853 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,854 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:15,854 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:15,854 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 02:54:15,855 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:15,855 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:15,856 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824885856 2023-02-08 02:54:15,857 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,857 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:15,859 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:15,860 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:15,860 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:15,860 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824915860 2023-02-08 02:54:15,860 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824975860 2023-02-08 02:54:15,860 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:15,863 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:15,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:15,866 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:15,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:15,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:16,111 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,111 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:16,111 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:16,111 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:16,113 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:16,113 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:16,113 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:16,113 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:16,113 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:16,114 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:16,115 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 02:54:16,115 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:16,115 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:16,116 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:16,116 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:16,116 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:16,116 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:16,117 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,117 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:16,117 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,118 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,118 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,118 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:16,118 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,119 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,119 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:16,120 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,120 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:16,120 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:16,120 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:16,120 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,120 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,120 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,120 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,121 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:16,122 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:16,122 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:16,123 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824886123 2023-02-08 02:54:16,125 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,125 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:16,128 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,128 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,129 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:16,129 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824916129 2023-02-08 02:54:16,129 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824976129 2023-02-08 02:54:16,129 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:54:16,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:16,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:16,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:16,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:16,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:16,380 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,380 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:16,380 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:16,380 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:16,381 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:16,381 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:16,381 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:16,381 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:16,381 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:16,382 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:16,383 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 02:54:16,383 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:16,383 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:16,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:16,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:16,386 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,386 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,386 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,386 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,387 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-08 02:54:16,387 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:16,388 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:16,388 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824886388 2023-02-08 02:54:16,391 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:16,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,393 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,394 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,394 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:16,394 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824916394 2023-02-08 02:54:16,394 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824976394 2023-02-08 02:54:16,394 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:16,394 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:16,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:16,398 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:16,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:16,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:16,645 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,645 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:16,645 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:16,645 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:16,646 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:16,647 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:16,647 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:16,647 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:16,647 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:16,647 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:16,648 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 02:54:16,648 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:16,649 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:16,649 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:16,649 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:16,649 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:16,650 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:16,650 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,650 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:16,650 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,651 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,651 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,651 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:16,651 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,652 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,652 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:16,653 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,653 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:16,653 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:16,653 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:16,653 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,653 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,653 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,654 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,654 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,654 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:16,654 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:16,655 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:16,655 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824886655 2023-02-08 02:54:16,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,657 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:16,659 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,659 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,659 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:16,660 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824916660 2023-02-08 02:54:16,660 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824976660 2023-02-08 02:54:16,660 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:16,660 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:16,662 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:16,662 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:16,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:16,662 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:16,910 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:16,910 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:16,911 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:16,912 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 02:54:16,912 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:16,912 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:16,912 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:16,912 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:16,914 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,914 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:16,914 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,914 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:16,914 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:16,915 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,915 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,915 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,915 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:16,915 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 02:54:16,916 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:16,917 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:16,919 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824886919 2023-02-08 02:54:16,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:16,921 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:16,924 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,924 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:16,924 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:16,924 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824916924 2023-02-08 02:54:16,924 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824976924 2023-02-08 02:54:16,924 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:16,924 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:16,929 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN] 2023-02-08 02:54:16,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:16,929 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:16,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:16,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:17,175 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:17,175 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:17,176 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:17,176 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:17,177 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 02:54:17,177 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:54:17,178 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:54:17,178 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:17,178 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:17,178 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:17,178 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:17,179 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:17,179 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,179 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:17,179 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,179 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,180 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,180 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:17,180 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,180 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,180 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:17,181 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:17,182 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,182 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,182 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,182 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,183 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-08 02:54:17,183 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:17,184 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:17,184 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:17,185 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824887184 2023-02-08 02:54:17,186 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:17,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,188 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,190 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,190 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,190 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:17,190 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824917190 2023-02-08 02:54:17,190 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824977190 2023-02-08 02:54:17,190 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 02:54:17,191 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:17,196 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:17,196 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:17,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:17,196 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:17,441 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,441 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:17,441 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:17,441 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:17,441 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:17,441 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:17,442 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:17,442 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:17,442 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:17,443 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:17,444 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:54:17,444 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:17,444 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:17,444 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:17,445 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:17,445 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:17,445 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,445 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:17,445 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,446 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,446 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,446 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:17,446 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,446 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,447 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:17,447 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,447 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,447 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:17,448 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,448 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,448 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,448 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,448 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:17,449 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:17,449 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:17,449 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:17,450 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824887450 2023-02-08 02:54:17,451 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:17,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,453 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,455 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,456 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,456 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:17,456 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824917456 2023-02-08 02:54:17,456 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824977456 2023-02-08 02:54:17,456 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 02:54:17,456 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:17,459 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:17,459 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:17,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:17,460 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:17,706 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,706 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:17,707 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:17,707 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:17,708 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:17,709 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 02:54:17,709 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:17,709 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:17,710 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:17,710 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:17,710 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:17,711 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,711 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:17,711 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,711 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,712 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,712 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:17,712 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,712 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,712 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:17,713 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,713 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,713 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:17,713 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,713 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,713 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,714 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,714 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-08 02:54:17,715 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:17,715 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:17,715 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:17,716 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824887716 2023-02-08 02:54:17,717 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:17,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,719 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,721 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,721 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,721 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:17,721 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824917721 2023-02-08 02:54:17,721 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824977721 2023-02-08 02:54:17,722 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 02:54:17,722 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:17,725 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:17,725 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-08 02:54:17,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:17,725 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:17,972 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:17,972 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:17,973 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:17,974 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 02:54:17,974 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-08 02:54:17,974 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:17,974 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,975 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:17,976 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,976 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:17,976 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,976 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:17,976 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:17,977 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,977 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,977 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:17,977 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,977 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,977 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,977 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:54:17,977 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 02:54:17,978 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:17,978 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:17,978 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:17,979 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824887979 2023-02-08 02:54:17,980 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:17,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:17,983 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,985 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,985 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:17,986 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:17,986 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824917986 2023-02-08 02:54:17,986 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824977986 2023-02-08 02:54:17,986 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-08 02:54:17,986 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:54:17,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 exec-time=5.2590 sec 2023-02-08 02:54:17,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:17,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=true 2023-02-08 02:54:18,140 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:18,141 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:18,141 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824858141"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824858141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824858141"}]},"ts":"1675824858141"} 2023-02-08 02:54:18,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:18,144 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:54:18,144 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:18,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:54:18,144 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:54:18,236 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:18,236 WARN [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-08 02:54:18,236 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:54:18,236 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:54:18,236 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:54:18,237 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:54:18,237 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:54:18,237 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:54:18,237 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:54:18,237 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:54:18,238 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:54:18,238 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-08 02:54:18,238 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:54:18,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:54:18,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:54:18,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:54:18,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:18,239 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:54:18,240 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:18,241 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:54:18,241 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:18,241 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:54:18,241 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:18,241 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-08 02:54:18,242 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:54:18,242 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:54:18,243 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876 to restore RIT 2023-02-08 02:54:18,243 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824888243 2023-02-08 02:54:18,244 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:54:18,244 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:54:18,246 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:18,246 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=a6003dd11a06ff730283e23c33e29876, regionState=OPENING, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:54:18,246 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:54:18,246 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824918246 2023-02-08 02:54:18,246 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675824978246 2023-02-08 02:54:18,246 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:54:18,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:18,247 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824858246"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824858246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824858246"}]},"ts":"1675824858246"} 2023-02-08 02:54:18,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:18,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6003dd11a06ff730283e23c33e29876, NAME => 'testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:18,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:18,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:18,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,407 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,408 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1 2023-02-08 02:54:18,408 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1 2023-02-08 02:54:18,408 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6003dd11a06ff730283e23c33e29876 columnFamilyName cf1 2023-02-08 02:54:18,415 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad 2023-02-08 02:54:18,419 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 2023-02-08 02:54:18,423 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e 2023-02-08 02:54:18,426 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b 2023-02-08 02:54:18,430 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 2023-02-08 02:54:18,434 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 2023-02-08 02:54:18,438 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 2023-02-08 02:54:18,438 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(310): Store=a6003dd11a06ff730283e23c33e29876/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:18,438 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,439 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2 2023-02-08 02:54:18,439 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2 2023-02-08 02:54:18,439 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6003dd11a06ff730283e23c33e29876 columnFamilyName cf2 2023-02-08 02:54:18,446 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb 2023-02-08 02:54:18,450 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 2023-02-08 02:54:18,454 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 2023-02-08 02:54:18,459 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 2023-02-08 02:54:18,463 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f 2023-02-08 02:54:18,469 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 2023-02-08 02:54:18,473 DEBUG [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb 2023-02-08 02:54:18,473 INFO [StoreOpener-a6003dd11a06ff730283e23c33e29876-1] regionserver.HStore(310): Store=a6003dd11a06ff730283e23c33e29876/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:18,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.splits 2023-02-08 02:54:18,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:18,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened a6003dd11a06ff730283e23c33e29876; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72848764, jitterRate=0.08553117513656616}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:18,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for a6003dd11a06ff730283e23c33e29876: Running coprocessor pre-open hook at 1675824858406Writing region info on filesystem at 1675824858406Initializing all the Stores at 1675824858406Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824858406Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824858406Cleaning up temporary data from old regions at 1675824858476 (+70 ms)Cleaning up detritus from prior splits at 1675824858476Running coprocessor post-open hooks at 1675824858480 (+4 ms)Region opened successfully at 1675824858480 2023-02-08 02:54:18,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876., pid=48, masterSystemTime=1675824858401 2023-02-08 02:54:18,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:18,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:18,482 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:18,482 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:18,483 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 02:54:18,483 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 02:54:18,483 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): a6003dd11a06ff730283e23c33e29876/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:18,483 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): a6003dd11a06ff730283e23c33e29876/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:18,483 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of a6003dd11a06ff730283e23c33e29876/cf2 in testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,483 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a6003dd11a06ff730283e23c33e29876/cf1 in testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,483 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp, totalSize=34.9 K 2023-02-08 02:54:18,484 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp, totalSize=34.9 K 2023-02-08 02:54:18,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,485 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting dd8e19326dbc4573a001176d728944cb, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824850766 2023-02-08 02:54:18,485 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting b689ce85b52f4fe5b49d4f9b88e2305b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824850766 2023-02-08 02:54:18,485 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 534653f171564dd8b8522cde21341881, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824851023 2023-02-08 02:54:18,485 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting b2bc62d9838445478da0cf9ec953a31e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824851023 2023-02-08 02:54:18,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 8d725120ea84482abd7e3a4fbdbdf10f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824851306 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting f3199f6bd66a4e309b8c65f65a54ee65, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824851306 2023-02-08 02:54:18,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824858485"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824858485"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824858485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824858485"}]},"ts":"1675824858485"} 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 82814f70319843b6a8229e958372e1d1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824851587 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 724a840295fe403eb58bd6fc1fcadf65, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824851587 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting b36a2620eced404599e093e53bb6cee0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824851822 2023-02-08 02:54:18,486 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting bb89df2970984436a5b2321f361082e5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824851822 2023-02-08 02:54:18,487 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 49f14693a2314106ad8cc7cc565d8ddb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824852127 2023-02-08 02:54:18,487 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 38691f4209a44316b0db128e76fdabad, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824852127 2023-02-08 02:54:18,487 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 6c138bb62d9c498bac4fca179071b865, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824852465 2023-02-08 02:54:18,487 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting d346a3302bb44494a1889ba354b94097, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824852465 2023-02-08 02:54:18,490 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-08 02:54:18,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 in 239 msec 2023-02-08 02:54:18,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN in 1.5630 sec 2023-02-08 02:54:18,508 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): a6003dd11a06ff730283e23c33e29876#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:18,509 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): a6003dd11a06ff730283e23c33e29876#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:18,529 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf2/d4932922a85c4a49bcb08f50b3f85a42 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/d4932922a85c4a49bcb08f50b3f85a42 2023-02-08 02:54:18,534 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a6003dd11a06ff730283e23c33e29876/cf2 of a6003dd11a06ff730283e23c33e29876 into d4932922a85c4a49bcb08f50b3f85a42(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:18,534 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:18,534 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876., storeName=a6003dd11a06ff730283e23c33e29876/cf2, priority=9, startTime=1675824858481; duration=0sec 2023-02-08 02:54:18,535 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:18,597 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-08 02:54:18,600 DEBUG [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-08 02:54:18,602 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 02:54:18,604 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-08 02:54:18,604 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-08 02:54:18,605 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-08 02:54:18,638 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:18,638 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:18,638 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:18,639 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:18,640 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:18,640 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-08 02:54:18,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackAndDoubleExecution 2023-02-08 02:54:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:18,644 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:18,644 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:18,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824858644"}]},"ts":"1675824858644"} 2023-02-08 02:54:18,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 02:54:18,646 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 02:54:18,745 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:18,745 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:18,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 02:54:18,761 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-08 02:54:18,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN}] 2023-02-08 02:54:18,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN 2023-02-08 02:54:18,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:18,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824858763"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824858763"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824858763"}]},"ts":"1675824858763"} 2023-02-08 02:54:18,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:18,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing a6003dd11a06ff730283e23c33e29876, disabling compactions & flushes 2023-02-08 02:54:18,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,931 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/.tmp/cf1/53d1747cce0448d0b7f74add729b0f2a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/53d1747cce0448d0b7f74add729b0f2a 2023-02-08 02:54:18,936 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a6003dd11a06ff730283e23c33e29876/cf1 of a6003dd11a06ff730283e23c33e29876 into 53d1747cce0448d0b7f74add729b0f2a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:18,937 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6003dd11a06ff730283e23c33e29876: 2023-02-08 02:54:18,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,937 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876., storeName=a6003dd11a06ff730283e23c33e29876/cf1, priority=9, startTime=1675824858481; duration=0sec 2023-02-08 02:54:18,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,937 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:18,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. after waiting 0 ms 2023-02-08 02:54:18,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,937 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097] to archive 2023-02-08 02:54:18,938 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:18,940 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b689ce85b52f4fe5b49d4f9b88e2305b 2023-02-08 02:54:18,941 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/b2bc62d9838445478da0cf9ec953a31e 2023-02-08 02:54:18,942 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/f3199f6bd66a4e309b8c65f65a54ee65 2023-02-08 02:54:18,944 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/724a840295fe403eb58bd6fc1fcadf65 2023-02-08 02:54:18,945 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/bb89df2970984436a5b2321f361082e5 2023-02-08 02:54:18,946 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/38691f4209a44316b0db128e76fdabad 2023-02-08 02:54:18,946 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:18,947 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:18,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 02:54:18,948 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/d346a3302bb44494a1889ba354b94097 2023-02-08 02:54:18,951 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865] to archive 2023-02-08 02:54:18,952 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:18,954 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/dd8e19326dbc4573a001176d728944cb 2023-02-08 02:54:18,955 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/534653f171564dd8b8522cde21341881 2023-02-08 02:54:18,955 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/8d725120ea84482abd7e3a4fbdbdf10f 2023-02-08 02:54:18,956 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/82814f70319843b6a8229e958372e1d1 2023-02-08 02:54:18,957 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/b36a2620eced404599e093e53bb6cee0 2023-02-08 02:54:18,958 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/49f14693a2314106ad8cc7cc565d8ddb 2023-02-08 02:54:18,959 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/6c138bb62d9c498bac4fca179071b865 2023-02-08 02:54:18,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:54:18,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:18,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876. 2023-02-08 02:54:18,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for a6003dd11a06ff730283e23c33e29876: Waiting for close lock at 1675824858917Running coprocessor pre-close hooks at 1675824858917Disabling compacts and flushes for region at 1675824858917Disabling writes for close at 1675824858937 (+20 ms)Writing region close event to WAL at 1675824858960 (+23 ms)Running coprocessor post-close hooks at 1675824858963 (+3 ms)Closed at 1675824858963 2023-02-08 02:54:18,965 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:18,965 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=a6003dd11a06ff730283e23c33e29876, regionState=CLOSED 2023-02-08 02:54:18,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824858965"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824858965"}]},"ts":"1675824858965"} 2023-02-08 02:54:18,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-08 02:54:18,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure a6003dd11a06ff730283e23c33e29876, server=jenkins-hbase12.apache.org,37605,1675824833879 in 203 msec 2023-02-08 02:54:18,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-08 02:54:18,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, UNASSIGN in 207 msec 2023-02-08 02:54:18,971 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824858971"}]},"ts":"1675824858971"} 2023-02-08 02:54:18,972 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 02:54:18,981 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-08 02:54:18,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 341 msec 2023-02-08 02:54:19,248 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:19,248 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:19,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 02:54:19,249 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-08 02:54:19,249 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:19,249 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:19,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackAndDoubleExecution 2023-02-08 02:54:19,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,253 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,253 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:19,253 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:19,253 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 02:54:19,257 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:19,259 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits] 2023-02-08 02:54:19,262 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/53d1747cce0448d0b7f74add729b0f2a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf1/53d1747cce0448d0b7f74add729b0f2a 2023-02-08 02:54:19,264 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/d4932922a85c4a49bcb08f50b3f85a42 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/cf2/d4932922a85c4a49bcb08f50b3f85a42 2023-02-08 02:54:19,267 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876/recovered.edits/90.seqid 2023-02-08 02:54:19,267 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackAndDoubleExecution/a6003dd11a06ff730283e23c33e29876 2023-02-08 02:54:19,267 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-08 02:54:19,271 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,273 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-08 02:54:19,275 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-08 02:54:19,276 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,276 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-08 02:54:19,276 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824859276"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:19,278 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 02:54:19,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a6003dd11a06ff730283e23c33e29876, NAME => 'testRollbackAndDoubleExecution,,1675824850257.a6003dd11a06ff730283e23c33e29876.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 02:54:19,278 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-08 02:54:19,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824859278"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:19,280 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-08 02:54:19,287 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:54:19,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 37 msec 2023-02-08 02:54:19,354 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:19,354 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:19,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-08 02:54:19,356 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-08 02:54:19,368 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401 (was 400) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46084 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44094 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48336 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase12:37605-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46124 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44112 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44070 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48350 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46092 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46120 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46144 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48330 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48320 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46104 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44086 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44096 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44054 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=858 (was 866), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=277 (was 247) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=3677 (was 3679) 2023-02-08 02:54:19,380 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=173, AvailableMemoryMB=3676 2023-02-08 02:54:19,380 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:19,380 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:19,380 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:19,392 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:54:19,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:54:19,394 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:19,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:54:19,397 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:54:19,398 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,399 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 empty. 2023-02-08 02:54:19,399 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,399 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 02:54:19,409 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:19,410 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7021d1e7a9deec290548f73330a10890, NAME => 'testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:54:19,420 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:19,421 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 7021d1e7a9deec290548f73330a10890, disabling compactions & flushes 2023-02-08 02:54:19,421 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,421 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,421 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. after waiting 0 ms 2023-02-08 02:54:19,421 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,421 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,421 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 7021d1e7a9deec290548f73330a10890: Waiting for close lock at 1675824859421Disabling compacts and flushes for region at 1675824859421Disabling writes for close at 1675824859421Writing region close event to WAL at 1675824859421Closed at 1675824859421 2023-02-08 02:54:19,423 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:54:19,424 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824859424"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824859424"}]},"ts":"1675824859424"} 2023-02-08 02:54:19,426 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:54:19,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:54:19,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824859427"}]},"ts":"1675824859427"} 2023-02-08 02:54:19,428 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-08 02:54:19,452 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:19,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:19,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:19,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:19,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:54:19,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, ASSIGN}] 2023-02-08 02:54:19,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, ASSIGN 2023-02-08 02:54:19,456 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46783,1675824833788; forceNewPlan=false, retain=false 2023-02-08 02:54:19,484 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:19,606 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:19,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=7021d1e7a9deec290548f73330a10890, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:19,608 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824859607"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824859607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824859607"}]},"ts":"1675824859607"} 2023-02-08 02:54:19,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 7021d1e7a9deec290548f73330a10890, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:19,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7021d1e7a9deec290548f73330a10890, NAME => 'testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:19,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:19,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:19,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,770 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,771 DEBUG [StoreOpener-7021d1e7a9deec290548f73330a10890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf1 2023-02-08 02:54:19,771 DEBUG [StoreOpener-7021d1e7a9deec290548f73330a10890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf1 2023-02-08 02:54:19,772 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7021d1e7a9deec290548f73330a10890 columnFamilyName cf1 2023-02-08 02:54:19,772 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] regionserver.HStore(310): Store=7021d1e7a9deec290548f73330a10890/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:19,772 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,774 DEBUG [StoreOpener-7021d1e7a9deec290548f73330a10890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf2 2023-02-08 02:54:19,774 DEBUG [StoreOpener-7021d1e7a9deec290548f73330a10890-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf2 2023-02-08 02:54:19,774 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7021d1e7a9deec290548f73330a10890 columnFamilyName cf2 2023-02-08 02:54:19,775 INFO [StoreOpener-7021d1e7a9deec290548f73330a10890-1] regionserver.HStore(310): Store=7021d1e7a9deec290548f73330a10890/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:19,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:19,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:19,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:54:19,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 7021d1e7a9deec290548f73330a10890; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60441984, jitterRate=-0.09934425354003906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:19,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 7021d1e7a9deec290548f73330a10890: Running coprocessor pre-open hook at 1675824859769Writing region info on filesystem at 1675824859769Initializing all the Stores at 1675824859770 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824859770Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824859770Cleaning up temporary data from old regions at 1675824859777 (+7 ms)Cleaning up detritus from prior splits at 1675824859777Running coprocessor post-open hooks at 1675824859783 (+6 ms)Region opened successfully at 1675824859783 2023-02-08 02:54:19,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890., pid=55, masterSystemTime=1675824859761 2023-02-08 02:54:19,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:19,786 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=7021d1e7a9deec290548f73330a10890, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:19,786 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824859786"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824859786"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824859786"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824859786"}]},"ts":"1675824859786"} 2023-02-08 02:54:19,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-08 02:54:19,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 7021d1e7a9deec290548f73330a10890, server=jenkins-hbase12.apache.org,46783,1675824833788 in 179 msec 2023-02-08 02:54:19,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-08 02:54:19,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, ASSIGN in 337 msec 2023-02-08 02:54:19,793 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:54:19,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824859793"}]},"ts":"1675824859793"} 2023-02-08 02:54:19,795 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-08 02:54:19,846 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:54:19,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 453 msec 2023-02-08 02:54:19,895 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:19,896 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:19,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:19,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:19,897 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:19,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, UNASSIGN}] 2023-02-08 02:54:19,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, UNASSIGN 2023-02-08 02:54:19,900 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=7021d1e7a9deec290548f73330a10890, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:19,900 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824859900"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824859900"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824859900"}]},"ts":"1675824859900"} 2023-02-08 02:54:19,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 7021d1e7a9deec290548f73330a10890, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:20,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:20,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 7021d1e7a9deec290548f73330a10890, disabling compactions & flushes 2023-02-08 02:54:20,056 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:20,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:20,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. after waiting 0 ms 2023-02-08 02:54:20,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:20,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 02:54:20,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:20,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890. 2023-02-08 02:54:20,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 7021d1e7a9deec290548f73330a10890: Waiting for close lock at 1675824860055Running coprocessor pre-close hooks at 1675824860055Disabling compacts and flushes for region at 1675824860055Disabling writes for close at 1675824860056 (+1 ms)Writing region close event to WAL at 1675824860056Running coprocessor post-close hooks at 1675824860060 (+4 ms)Closed at 1675824860060 2023-02-08 02:54:20,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:20,063 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=7021d1e7a9deec290548f73330a10890, regionState=CLOSED 2023-02-08 02:54:20,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824860063"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824860063"}]},"ts":"1675824860063"} 2023-02-08 02:54:20,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-08 02:54:20,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 7021d1e7a9deec290548f73330a10890, server=jenkins-hbase12.apache.org,46783,1675824833788 in 163 msec 2023-02-08 02:54:20,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-08 02:54:20,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7021d1e7a9deec290548f73330a10890, UNASSIGN in 168 msec 2023-02-08 02:54:20,109 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 02:54:20,111 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-08 02:54:20,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675824860112"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675824860112"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675824860112"}]},"ts":"1675824860112"} 2023-02-08 02:54:20,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860112"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824860112"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824860112"}]},"ts":"1675824860112"} 2023-02-08 02:54:20,113 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860112"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824860112"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824860112"}]},"ts":"1675824860112"} 2023-02-08 02:54:20,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, ASSIGN}] 2023-02-08 02:54:20,118 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, ASSIGN 2023-02-08 02:54:20,118 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46783,1675824833788; forceNewPlan=false, retain=false 2023-02-08 02:54:20,119 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, ASSIGN 2023-02-08 02:54:20,120 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,46783,1675824833788; forceNewPlan=false, retain=false 2023-02-08 02:54:20,269 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:54:20,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=d4cc7e9a06f1fc94b8cfc192b9959729, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,271 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860270"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824860270"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824860270"}]},"ts":"1675824860270"} 2023-02-08 02:54:20,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure d4cc7e9a06f1fc94b8cfc192b9959729, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:20,275 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=856866739a370df8d72a12d9a5e896ab, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,275 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860275"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824860275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824860275"}]},"ts":"1675824860275"} 2023-02-08 02:54:20,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 856866739a370df8d72a12d9a5e896ab, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:20,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 856866739a370df8d72a12d9a5e896ab, NAME => 'testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 02:54:20,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:20,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:20,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,429 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,431 DEBUG [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf1 2023-02-08 02:54:20,431 DEBUG [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf1 2023-02-08 02:54:20,431 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 856866739a370df8d72a12d9a5e896ab columnFamilyName cf1 2023-02-08 02:54:20,432 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] regionserver.HStore(310): Store=856866739a370df8d72a12d9a5e896ab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:20,432 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,433 DEBUG [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf2 2023-02-08 02:54:20,433 DEBUG [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf2 2023-02-08 02:54:20,433 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 856866739a370df8d72a12d9a5e896ab columnFamilyName cf2 2023-02-08 02:54:20,434 INFO [StoreOpener-856866739a370df8d72a12d9a5e896ab-1] regionserver.HStore(310): Store=856866739a370df8d72a12d9a5e896ab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:20,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:20,438 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 856866739a370df8d72a12d9a5e896ab; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71299147, jitterRate=0.06244008243083954}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:20,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 856866739a370df8d72a12d9a5e896ab: Running coprocessor pre-open hook at 1675824860428Writing region info on filesystem at 1675824860428Initializing all the Stores at 1675824860429 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824860429Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824860429Cleaning up temporary data from old regions at 1675824860436 (+7 ms)Cleaning up detritus from prior splits at 1675824860437 (+1 ms)Running coprocessor post-open hooks at 1675824860439 (+2 ms)Region opened successfully at 1675824860439 2023-02-08 02:54:20,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab., pid=62, masterSystemTime=1675824860424 2023-02-08 02:54:20,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d4cc7e9a06f1fc94b8cfc192b9959729, NAME => 'testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 02:54:20,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=856866739a370df8d72a12d9a5e896ab, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860441"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824860441"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824860441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824860441"}]},"ts":"1675824860441"} 2023-02-08 02:54:20,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:20,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:20,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,443 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,445 DEBUG [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf1 2023-02-08 02:54:20,445 DEBUG [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf1 2023-02-08 02:54:20,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-08 02:54:20,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 856866739a370df8d72a12d9a5e896ab, server=jenkins-hbase12.apache.org,46783,1675824833788 in 166 msec 2023-02-08 02:54:20,445 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4cc7e9a06f1fc94b8cfc192b9959729 columnFamilyName cf1 2023-02-08 02:54:20,446 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] regionserver.HStore(310): Store=d4cc7e9a06f1fc94b8cfc192b9959729/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:20,446 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, ASSIGN in 329 msec 2023-02-08 02:54:20,447 DEBUG [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf2 2023-02-08 02:54:20,447 DEBUG [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf2 2023-02-08 02:54:20,447 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4cc7e9a06f1fc94b8cfc192b9959729 columnFamilyName cf2 2023-02-08 02:54:20,448 INFO [StoreOpener-d4cc7e9a06f1fc94b8cfc192b9959729-1] regionserver.HStore(310): Store=d4cc7e9a06f1fc94b8cfc192b9959729/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:20,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:20,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened d4cc7e9a06f1fc94b8cfc192b9959729; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60172092, jitterRate=-0.103365957736969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:20,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for d4cc7e9a06f1fc94b8cfc192b9959729: Running coprocessor pre-open hook at 1675824860442Writing region info on filesystem at 1675824860442Initializing all the Stores at 1675824860443 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824860443Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824860443Cleaning up temporary data from old regions at 1675824860450 (+7 ms)Cleaning up detritus from prior splits at 1675824860450Running coprocessor post-open hooks at 1675824860453 (+3 ms)Region opened successfully at 1675824860453 2023-02-08 02:54:20,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729., pid=61, masterSystemTime=1675824860424 2023-02-08 02:54:20,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,456 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=d4cc7e9a06f1fc94b8cfc192b9959729, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,456 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860456"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824860456"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824860456"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824860456"}]},"ts":"1675824860456"} 2023-02-08 02:54:20,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-08 02:54:20,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure d4cc7e9a06f1fc94b8cfc192b9959729, server=jenkins-hbase12.apache.org,46783,1675824833788 in 186 msec 2023-02-08 02:54:20,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-08 02:54:20,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, ASSIGN in 344 msec 2023-02-08 02:54:20,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab in 566 msec 2023-02-08 02:54:20,651 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:54:20,654 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:20,654 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,654 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,655 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,655 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,656 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,659 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,659 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824860659"}]},"ts":"1675824860659"} 2023-02-08 02:54:20,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 02:54:20,660 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-08 02:54:20,718 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-08 02:54:20,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, UNASSIGN}] 2023-02-08 02:54:20,720 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, UNASSIGN 2023-02-08 02:54:20,721 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=856866739a370df8d72a12d9a5e896ab, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,721 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860721"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824860721"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824860721"}]},"ts":"1675824860721"} 2023-02-08 02:54:20,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 856866739a370df8d72a12d9a5e896ab, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:20,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, UNASSIGN 2023-02-08 02:54:20,725 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=d4cc7e9a06f1fc94b8cfc192b9959729, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:20,726 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860725"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824860725"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824860725"}]},"ts":"1675824860725"} 2023-02-08 02:54:20,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure d4cc7e9a06f1fc94b8cfc192b9959729, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:20,760 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,760 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 02:54:20,875 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing d4cc7e9a06f1fc94b8cfc192b9959729, disabling compactions & flushes 2023-02-08 02:54:20,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. after waiting 0 ms 2023-02-08 02:54:20,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 02:54:20,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:20,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729. 2023-02-08 02:54:20,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for d4cc7e9a06f1fc94b8cfc192b9959729: Waiting for close lock at 1675824860875Running coprocessor pre-close hooks at 1675824860875Disabling compacts and flushes for region at 1675824860875Disabling writes for close at 1675824860876 (+1 ms)Writing region close event to WAL at 1675824860876Running coprocessor post-close hooks at 1675824860880 (+4 ms)Closed at 1675824860880 2023-02-08 02:54:20,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 856866739a370df8d72a12d9a5e896ab, disabling compactions & flushes 2023-02-08 02:54:20,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. after waiting 0 ms 2023-02-08 02:54:20,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,883 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=d4cc7e9a06f1fc94b8cfc192b9959729, regionState=CLOSED 2023-02-08 02:54:20,883 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860883"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824860883"}]},"ts":"1675824860883"} 2023-02-08 02:54:20,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 02:54:20,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-08 02:54:20,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure d4cc7e9a06f1fc94b8cfc192b9959729, server=jenkins-hbase12.apache.org,46783,1675824833788 in 158 msec 2023-02-08 02:54:20,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:20,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab. 2023-02-08 02:54:20,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 856866739a370df8d72a12d9a5e896ab: Waiting for close lock at 1675824860882Running coprocessor pre-close hooks at 1675824860882Disabling compacts and flushes for region at 1675824860882Disabling writes for close at 1675824860883 (+1 ms)Writing region close event to WAL at 1675824860883Running coprocessor post-close hooks at 1675824860888 (+5 ms)Closed at 1675824860888 2023-02-08 02:54:20,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d4cc7e9a06f1fc94b8cfc192b9959729, UNASSIGN in 169 msec 2023-02-08 02:54:20,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,891 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=856866739a370df8d72a12d9a5e896ab, regionState=CLOSED 2023-02-08 02:54:20,891 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824860891"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824860891"}]},"ts":"1675824860891"} 2023-02-08 02:54:20,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-08 02:54:20,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 856866739a370df8d72a12d9a5e896ab, server=jenkins-hbase12.apache.org,46783,1675824833788 in 169 msec 2023-02-08 02:54:20,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-08 02:54:20,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=856866739a370df8d72a12d9a5e896ab, UNASSIGN in 175 msec 2023-02-08 02:54:20,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824860897"}]},"ts":"1675824860897"} 2023-02-08 02:54:20,898 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-08 02:54:20,908 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-08 02:54:20,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 253 msec 2023-02-08 02:54:20,961 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,961 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-08 02:54:20,962 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-08 02:54:20,962 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,962 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,966 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,966 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:20,966 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:20,967 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 02:54:20,972 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:20,972 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,972 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,974 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/recovered.edits] 2023-02-08 02:54:20,974 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/recovered.edits] 2023-02-08 02:54:20,974 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/recovered.edits] 2023-02-08 02:54:20,980 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/recovered.edits/7.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729/recovered.edits/7.seqid 2023-02-08 02:54:20,980 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/recovered.edits/4.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890/recovered.edits/4.seqid 2023-02-08 02:54:20,980 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/recovered.edits/7.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab/recovered.edits/7.seqid 2023-02-08 02:54:20,981 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/d4cc7e9a06f1fc94b8cfc192b9959729 2023-02-08 02:54:20,981 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/7021d1e7a9deec290548f73330a10890 2023-02-08 02:54:20,981 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionNoStoreFile/856866739a370df8d72a12d9a5e896ab 2023-02-08 02:54:20,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-08 02:54:20,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,985 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-08 02:54:20,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-08 02:54:20,988 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:20,988 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-08 02:54:20,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824860988"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:20,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824860988"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:20,988 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824860988"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:20,990 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:54:20,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7021d1e7a9deec290548f73330a10890, NAME => 'testSplitTableRegionNoStoreFile,,1675824859393.7021d1e7a9deec290548f73330a10890.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d4cc7e9a06f1fc94b8cfc192b9959729, NAME => 'testSplitTableRegionNoStoreFile,,1675824859895.d4cc7e9a06f1fc94b8cfc192b9959729.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 856866739a370df8d72a12d9a5e896ab, NAME => 'testSplitTableRegionNoStoreFile,41,1675824859895.856866739a370df8d72a12d9a5e896ab.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 02:54:20,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-08 02:54:20,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824860990"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:20,992 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-08 02:54:21,003 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:54:21,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 40 msec 2023-02-08 02:54:21,067 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:21,067 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:21,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-08 02:54:21,068 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-08 02:54:21,080 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=408 (was 401) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_507276565_15 at /127.0.0.1:44070 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_507276565_15 at /127.0.0.1:44054 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=858 (was 858), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=277 (was 277), ProcessCount=173 (was 173), AvailableMemoryMB=3628 (was 3676) 2023-02-08 02:54:21,092 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=173, AvailableMemoryMB=3627 2023-02-08 02:54:21,093 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:21,093 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:21,093 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:21,104 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:54:21,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:54:21,106 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,107 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:54:21,108 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:54:21,109 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,110 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 empty. 2023-02-08 02:54:21,110 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,110 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 02:54:21,123 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:21,124 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 833776ad2367c20696c3f51f3ecac4a3, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:54:21,137 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:21,138 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 833776ad2367c20696c3f51f3ecac4a3, disabling compactions & flushes 2023-02-08 02:54:21,138 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,138 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,138 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. after waiting 0 ms 2023-02-08 02:54:21,138 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,138 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,138 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 833776ad2367c20696c3f51f3ecac4a3: Waiting for close lock at 1675824861138Disabling compacts and flushes for region at 1675824861138Disabling writes for close at 1675824861138Writing region close event to WAL at 1675824861138Closed at 1675824861138 2023-02-08 02:54:21,140 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:54:21,140 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824861140"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824861140"}]},"ts":"1675824861140"} 2023-02-08 02:54:21,142 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:54:21,143 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:54:21,143 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824861143"}]},"ts":"1675824861143"} 2023-02-08 02:54:21,144 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-08 02:54:21,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:21,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:21,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:21,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:21,168 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:54:21,169 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, ASSIGN}] 2023-02-08 02:54:21,170 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, ASSIGN 2023-02-08 02:54:21,171 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:54:21,321 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:21,322 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=833776ad2367c20696c3f51f3ecac4a3, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,322 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824861322"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824861322"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824861322"}]},"ts":"1675824861322"} 2023-02-08 02:54:21,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 833776ad2367c20696c3f51f3ecac4a3, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:21,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 833776ad2367c20696c3f51f3ecac4a3, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:21,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:21,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:21,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,480 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,482 DEBUG [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1 2023-02-08 02:54:21,482 DEBUG [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1 2023-02-08 02:54:21,482 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 833776ad2367c20696c3f51f3ecac4a3 columnFamilyName cf1 2023-02-08 02:54:21,482 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] regionserver.HStore(310): Store=833776ad2367c20696c3f51f3ecac4a3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:21,483 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,484 DEBUG [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2 2023-02-08 02:54:21,484 DEBUG [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2 2023-02-08 02:54:21,484 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 833776ad2367c20696c3f51f3ecac4a3 columnFamilyName cf2 2023-02-08 02:54:21,485 INFO [StoreOpener-833776ad2367c20696c3f51f3ecac4a3-1] regionserver.HStore(310): Store=833776ad2367c20696c3f51f3ecac4a3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:21,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:21,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:21,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:54:21,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 833776ad2367c20696c3f51f3ecac4a3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69554173, jitterRate=0.03643794357776642}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:21,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 833776ad2367c20696c3f51f3ecac4a3: Running coprocessor pre-open hook at 1675824861479Writing region info on filesystem at 1675824861479Initializing all the Stores at 1675824861480 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824861480Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824861480Cleaning up temporary data from old regions at 1675824861486 (+6 ms)Cleaning up detritus from prior splits at 1675824861487 (+1 ms)Running coprocessor post-open hooks at 1675824861492 (+5 ms)Region opened successfully at 1675824861492 2023-02-08 02:54:21,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3., pid=71, masterSystemTime=1675824861476 2023-02-08 02:54:21,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,495 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=833776ad2367c20696c3f51f3ecac4a3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824861495"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824861495"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824861495"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824861495"}]},"ts":"1675824861495"} 2023-02-08 02:54:21,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-08 02:54:21,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 833776ad2367c20696c3f51f3ecac4a3, server=jenkins-hbase12.apache.org,37605,1675824833879 in 173 msec 2023-02-08 02:54:21,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-08 02:54:21,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, ASSIGN in 329 msec 2023-02-08 02:54:21,501 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:54:21,501 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824861501"}]},"ts":"1675824861501"} 2023-02-08 02:54:21,502 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-08 02:54:21,516 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:54:21,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 411 msec 2023-02-08 02:54:21,614 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:21,614 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:21,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:21,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,617 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:21,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:21,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:21,618 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,618 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,623 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,623 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,623 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,623 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,624 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,624 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:21,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:21,626 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,625 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,626 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:21,626 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,626 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,627 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:54:21,627 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,634 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,634 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,639 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,639 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,644 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/2904fbb41ea54a799fb4538e24ddd0ea 2023-02-08 02:54:21,667 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/75225cc3506a43a18222d00b51096438 2023-02-08 02:54:21,672 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/2904fbb41ea54a799fb4538e24ddd0ea as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea 2023-02-08 02:54:21,676 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:21,677 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/75225cc3506a43a18222d00b51096438 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438 2023-02-08 02:54:21,683 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:54:21,684 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 57ms, sequenceid=6, compaction requested=false 2023-02-08 02:54:21,684 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:21,684 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:21,684 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,685 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:21,685 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:21,685 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,685 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,685 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,697 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,697 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:21,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:21,697 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,697 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:21,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ff9c11f[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:21,699 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:21,699 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:21,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,708 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:21,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,708 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:21,718 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:21,718 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,718 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:21,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,729 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:21,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:21,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:21,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:21,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:21,732 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:21,732 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:21,732 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:21,732 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,732 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:21,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:21,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,742 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,742 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:21,742 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,778 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:21,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:21,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:21,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,779 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,780 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,780 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,780 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:21,780 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:21,880 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:21,880 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:21,880 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:21,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:21,895 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:21,896 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:21,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:21,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:21,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,898 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:21,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:21,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:21,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,918 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,918 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,918 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,918 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:21,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:21,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,920 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:21,920 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:21,920 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:21,921 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:21,928 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,928 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,929 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/7503e7228a3c4afe92c95cd6da8b6089 2023-02-08 02:54:21,929 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,929 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,929 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,929 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,940 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/170989de44814a738f48f47ba3dd7c86 2023-02-08 02:54:21,944 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/7503e7228a3c4afe92c95cd6da8b6089 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089 2023-02-08 02:54:21,947 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:21,948 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/170989de44814a738f48f47ba3dd7c86 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86 2023-02-08 02:54:21,952 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:54:21,953 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 32ms, sequenceid=19, compaction requested=false 2023-02-08 02:54:21,953 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:21,953 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:21,953 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:21,953 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:21,953 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,953 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:21,953 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:21,960 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,960 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:21,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:21,961 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,961 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bed8688[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:21,962 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:21,962 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:21,971 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,971 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,971 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:21,971 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,971 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:21,972 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:21,972 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:21,972 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:21,972 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:21,981 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:21,981 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,981 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:21,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:21,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:21,982 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:21,992 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:21,992 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:21,992 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,002 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,002 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,002 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:22,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:22,006 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:22,006 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:22,006 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:22,006 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,006 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,015 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,015 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,016 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,025 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,124 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:22,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:22,138 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,138 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:22,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:22,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,140 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,140 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,140 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:22,140 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:22,141 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,141 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,150 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,150 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,150 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,151 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,151 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,152 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,152 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:22,152 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,152 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:22,162 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/9efc60edeee5492bbb97be409fe8cbe8 2023-02-08 02:54:22,172 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/85fca2aa40a64420a233e048f3e9330d 2023-02-08 02:54:22,176 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/9efc60edeee5492bbb97be409fe8cbe8 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8 2023-02-08 02:54:22,181 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:22,182 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/85fca2aa40a64420a233e048f3e9330d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d 2023-02-08 02:54:22,186 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:54:22,186 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 34ms, sequenceid=32, compaction requested=true 2023-02-08 02:54:22,187 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:22,187 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,187 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:22,187 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:22,187 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,187 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,187 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,211 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,211 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,230 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,231 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,230 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,231 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,230 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,231 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,231 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:22,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e562b52[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:22,233 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:22,233 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,245 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:22,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,245 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:22,255 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,255 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,255 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,266 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,266 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:22,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:22,269 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:22,269 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:22,269 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:22,269 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,269 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,277 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,277 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,277 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,279 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,279 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,279 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,287 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,287 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,288 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,388 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:22,388 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,388 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:22,403 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,403 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:22,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:22,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,406 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:22,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:22,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,406 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,413 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,413 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,413 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:22,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,415 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,415 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,416 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:22,416 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,416 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:22,416 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:22,416 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:22,423 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,424 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/a50c9e5096fe4444a270d651d46fb35a 2023-02-08 02:54:22,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,434 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,434 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,434 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,435 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/dd1e7cd5c11a4648936e9073a5759e60 2023-02-08 02:54:22,439 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/a50c9e5096fe4444a270d651d46fb35a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a 2023-02-08 02:54:22,444 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:22,445 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/dd1e7cd5c11a4648936e9073a5759e60 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60 2023-02-08 02:54:22,449 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:54:22,450 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 34ms, sequenceid=45, compaction requested=true 2023-02-08 02:54:22,450 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:22,450 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:22,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:22,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,462 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,462 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,463 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,463 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:22,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@747951cf[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:22,465 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:22,465 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,476 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:22,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,476 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,476 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:22,477 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,477 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,477 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:22,476 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,477 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,477 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:22,487 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,487 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,487 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,528 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,528 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,528 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,529 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,529 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,529 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,529 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:22,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:22,532 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:22,532 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:22,532 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:22,532 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,532 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,551 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,551 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,552 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,553 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,553 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,560 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,561 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,562 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,562 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,562 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,562 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,661 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:22,661 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,661 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:22,674 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:22,674 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:22,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:22,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:22,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,676 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,676 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,676 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:22,676 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:22,677 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,677 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,687 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,687 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:22,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,688 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,689 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:22,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,689 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,689 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:22,689 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,689 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,689 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:22,697 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,698 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/850f59ce6e014ba7a92508246a5d346f 2023-02-08 02:54:22,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,708 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,708 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,708 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,716 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/567e05b7199b44039a115ae10b522ca5 2023-02-08 02:54:22,720 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/850f59ce6e014ba7a92508246a5d346f as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f 2023-02-08 02:54:22,725 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:22,725 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/567e05b7199b44039a115ae10b522ca5 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5 2023-02-08 02:54:22,729 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:54:22,730 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 41ms, sequenceid=58, compaction requested=true 2023-02-08 02:54:22,730 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:22,730 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:22,730 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:22,730 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:22,730 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,730 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:22,730 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:22,739 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:22,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:22,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5798766[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:22,741 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:22,741 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,750 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,750 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:22,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:22,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:22,760 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,760 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,760 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,809 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,810 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,810 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,809 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:22,810 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,810 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,810 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:22,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:22,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:22,813 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:22,813 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:22,813 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:22,813 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,813 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:22,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,898 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,899 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:22,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,899 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,899 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,900 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,901 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:22,901 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,901 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:22,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:22,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:22,973 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:22,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:22,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:22,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:22,973 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:22,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:22,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,074 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:23,074 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,074 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:23,086 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,086 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:23,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:23,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,089 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,089 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,089 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:23,089 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:23,089 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,089 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,108 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,108 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,108 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,108 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,110 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,110 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:23,109 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,110 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:23,110 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:23,118 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,125 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/2086baacc7a348149441de014ac9a43e 2023-02-08 02:54:23,128 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,128 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,129 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,129 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,129 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,137 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/38521ca7fcfd4e60961875241269f48a 2023-02-08 02:54:23,141 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/2086baacc7a348149441de014ac9a43e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e 2023-02-08 02:54:23,144 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:23,145 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/38521ca7fcfd4e60961875241269f48a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a 2023-02-08 02:54:23,148 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:54:23,149 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 39ms, sequenceid=71, compaction requested=true 2023-02-08 02:54:23,149 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:23,149 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,149 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:23,149 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:23,149 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,149 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,150 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,160 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,160 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,161 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,161 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:23,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2279887[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:23,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:23,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,171 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,171 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:23,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:23,171 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,172 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:23,172 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,172 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:23,181 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,181 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,181 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,181 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,192 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:23,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:23,195 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:23,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:23,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:23,195 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,195 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,277 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,278 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,279 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,280 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,280 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,280 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:23,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,308 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:23,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:23,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,308 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,309 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,408 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:23,409 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,409 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:23,421 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,421 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:23,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:23,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,423 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,423 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,423 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:23,423 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:23,424 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,424 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,434 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,434 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:23,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,436 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,437 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,437 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,437 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,437 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,437 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,437 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,439 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:23,439 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:23,439 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,439 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:23,439 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-08 02:54:23,444 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,447 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:23,451 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:23,455 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,455 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,455 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,455 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,455 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,458 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:23,461 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:23,462 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/7e3e270c82174471803086e830f32b0c as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:23,466 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:23,466 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 02:54:23,467 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/bd203221148e46fabfe184e82202baf8 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:23,470 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:23,470 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8, entries=9, sequenceid=83, filesize=5.1 K 2023-02-08 02:54:23,471 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 32ms, sequenceid=83, compaction requested=true 2023-02-08 02:54:23,471 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:23,472 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,472 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:23,472 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:23,472 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,472 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,472 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,483 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,483 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,484 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,484 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:23,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dd43f2a[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:23,486 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:23,486 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,497 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,497 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:23,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,497 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:23,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,498 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,498 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:23,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,550 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,550 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,550 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,560 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,560 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,560 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:23,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:23,563 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:23,563 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:23,563 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:23,563 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,563 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,572 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,572 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,573 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,573 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,574 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,581 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,581 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,581 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,581 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,581 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,582 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:23,582 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,583 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,683 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:23,683 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,683 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:23,692 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,692 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:23,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:23,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,694 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,694 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,694 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:23,694 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:23,695 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,695 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,704 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,704 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,704 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,704 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,705 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,705 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,706 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:23,707 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:23,706 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,706 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:23,707 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 02:54:23,715 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:23,718 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,718 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:23,726 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:23,728 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,728 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,729 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,729 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,729 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,729 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:23,730 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/0f448898212947b2a8cd1418ea869277 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:23,734 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:23,734 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 02:54:23,735 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/00c3af6c67344a51b3da13bf09842ea4 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:23,739 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:23,739 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4, entries=5, sequenceid=91, filesize=5.0 K 2023-02-08 02:54:23,740 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 33ms, sequenceid=91, compaction requested=true 2023-02-08 02:54:23,740 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:23,740 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:23,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:23,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,750 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,750 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:23,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@431dee76[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:23,752 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:23,752 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,760 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:23,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:23,761 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:23,771 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,771 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,771 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,781 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,781 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,781 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:23,781 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,781 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,782 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,781 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:23,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:23,785 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:23,785 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:23,785 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:23,785 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,785 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:23,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:23,792 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,792 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,793 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,793 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:23,794 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,794 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,795 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,829 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,829 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:23,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:23,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,830 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,930 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:23,931 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,931 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:23,939 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:23,939 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:23,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:54:23,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:54:23,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,941 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:23,941 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,941 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-08 02:54:23,941 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-08 02:54:23,942 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,942 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,960 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,960 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,960 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,960 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:23,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:23,961 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,961 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:54:23,962 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:54:23,962 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,963 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. started... 2023-02-08 02:54:23,963 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,971 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:23,973 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:23,977 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:23,985 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:23,988 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:23,989 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/0b7bc862d7724752aea437fa6606c4dd as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:23,993 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:23,993 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 02:54:23,994 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/419b49e58f894c0084eb863dc918da8b as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:23,998 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:23,998 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b, entries=5, sequenceid=99, filesize=5.0 K 2023-02-08 02:54:23,999 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 36ms, sequenceid=99, compaction requested=true 2023-02-08 02:54:23,999 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 833776ad2367c20696c3f51f3ecac4a3: 2023-02-08 02:54:23,999 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:23,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:54:23,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:54:23,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:23,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-08 02:54:23,999 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-08 02:54:24,010 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,010 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:24,010 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,010 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:54:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:24,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-08 02:54:24,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a9613c4[Count = 0] remaining members to acquire global barrier 2023-02-08 02:54:24,012 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-08 02:54:24,012 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:24,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:54:24,024 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,024 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-08 02:54:24,024 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:24,034 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:24,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:24,034 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:24,044 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-08 02:54:24,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:24,044 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:24,045 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-08 02:54:24,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:24,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:54:24,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:54:24,047 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-08 02:54:24,047 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:54:24,047 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:54:24,047 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,047 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:54:24,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,055 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,056 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,058 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,058 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:54:24,058 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,065 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,065 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-08 02:54:24,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:54:24,065 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,066 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:54:24,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,066 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:54:24,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,067 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:24,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:54:24,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:54:24,167 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:54:24,167 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:24,167 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:24,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:54:24,170 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,171 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:24,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:24,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:24,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:24,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, UNASSIGN}] 2023-02-08 02:54:24,174 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, UNASSIGN 2023-02-08 02:54:24,175 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=833776ad2367c20696c3f51f3ecac4a3, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:24,175 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824864175"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824864175"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824864175"}]},"ts":"1675824864175"} 2023-02-08 02:54:24,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 833776ad2367c20696c3f51f3ecac4a3, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:24,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 833776ad2367c20696c3f51f3ecac4a3, disabling compactions & flushes 2023-02-08 02:54:24,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:24,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:24,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. after waiting 0 ms 2023-02-08 02:54:24,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:24,328 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 833776ad2367c20696c3f51f3ecac4a3 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-08 02:54:24,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf1/536a6952ab494b32a1cf57f68c09645b as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 02:54:24,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.tmp/cf2/4f79a8f8588e4507a0adb58078c961b9 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9, entries=1, sequenceid=103, filesize=4.9 K 2023-02-08 02:54:24,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 39ms, sequenceid=103, compaction requested=true 2023-02-08 02:54:24,372 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-08 02:54:24,372 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:24,372 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. 2023-02-08 02:54:24,372 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 833776ad2367c20696c3f51f3ecac4a3: Waiting for close lock at 1675824864328Running coprocessor pre-close hooks at 1675824864328Disabling compacts and flushes for region at 1675824864328Disabling writes for close at 1675824864328Obtaining lock to block concurrent updates at 1675824864328Preparing flush snapshotting stores in 833776ad2367c20696c3f51f3ecac4a3 at 1675824864328Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675824864329 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3. at 1675824864329Flushing 833776ad2367c20696c3f51f3ecac4a3/cf1: creating writer at 1675824864330 (+1 ms)Flushing 833776ad2367c20696c3f51f3ecac4a3/cf1: appending metadata at 1675824864332 (+2 ms)Flushing 833776ad2367c20696c3f51f3ecac4a3/cf1: closing flushed file at 1675824864332Flushing 833776ad2367c20696c3f51f3ecac4a3/cf2: creating writer at 1675824864343 (+11 ms)Flushing 833776ad2367c20696c3f51f3ecac4a3/cf2: appending metadata at 1675824864345 (+2 ms)Flushing 833776ad2367c20696c3f51f3ecac4a3/cf2: closing flushed file at 1675824864345Flushing 833776ad2367c20696c3f51f3ecac4a3/cf1: reopening flushed file at 1675824864358 (+13 ms)Flushing 833776ad2367c20696c3f51f3ecac4a3/cf2: reopening flushed file at 1675824864363 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 833776ad2367c20696c3f51f3ecac4a3 in 39ms, sequenceid=103, compaction requested=true at 1675824864367 (+4 ms)Writing region close event to WAL at 1675824864369 (+2 ms)Running coprocessor post-close hooks at 1675824864372 (+3 ms)Closed at 1675824864372 2023-02-08 02:54:24,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=833776ad2367c20696c3f51f3ecac4a3, regionState=CLOSED 2023-02-08 02:54:24,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824864374"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824864374"}]},"ts":"1675824864374"} 2023-02-08 02:54:24,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-08 02:54:24,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 833776ad2367c20696c3f51f3ecac4a3, server=jenkins-hbase12.apache.org,37605,1675824833879 in 200 msec 2023-02-08 02:54:24,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-08 02:54:24,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=833776ad2367c20696c3f51f3ecac4a3, UNASSIGN in 203 msec 2023-02-08 02:54:24,390 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=833776ad2367c20696c3f51f3ecac4a3, threads=16 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,391 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,392 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,393 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,398 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,398 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,398 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,399 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,399 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,401 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,401 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,402 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,409 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:24,412 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:24,412 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e, top=false 2023-02-08 02:54:24,413 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a, top=false 2023-02-08 02:54:24,412 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5, top=false 2023-02-08 02:54:24,414 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:24,414 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,417 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60, top=false 2023-02-08 02:54:24,417 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,420 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:24,424 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:24,426 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:24,426 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4, top=true 2023-02-08 02:54:24,427 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:24,427 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b, top=true 2023-02-08 02:54:24,429 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:24,429 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9, top=true 2023-02-08 02:54:24,430 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:24,430 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b, top=true 2023-02-08 02:54:24,432 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:24,432 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd, top=true 2023-02-08 02:54:24,445 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:24,446 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277, top=true 2023-02-08 02:54:24,461 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,461 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,461 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,462 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9 for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,462 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,462 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,464 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f, top=false 2023-02-08 02:54:24,464 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:24,466 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:24,467 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60 for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,474 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5 for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,475 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,475 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,478 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,478 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,493 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,496 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,496 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,496 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,504 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,505 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,505 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4 for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,505 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,505 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,510 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,513 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277 for child: 2a409a2959c68e7714a42d395c76489c, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,513 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,528 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,542 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a, top=false 2023-02-08 02:54:24,543 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,557 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/.splits/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a for child: b1f619421cec5a6184b63bb1631f4a72, parent: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,560 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:24,561 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,581 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:24,589 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,597 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,598 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,612 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,624 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,625 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8 for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:24,631 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,001 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d for region: 833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,001 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 833776ad2367c20696c3f51f3ecac4a3 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:54:25,045 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 02:54:25,047 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-08 02:54:25,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675824865049"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675824865049"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675824865049"}]},"ts":"1675824865049"} 2023-02-08 02:54:25,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824865049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824865049"}]},"ts":"1675824865049"} 2023-02-08 02:54:25,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824865049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824865049"}]},"ts":"1675824865049"} 2023-02-08 02:54:25,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, ASSIGN}] 2023-02-08 02:54:25,053 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, ASSIGN 2023-02-08 02:54:25,054 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:54:25,054 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, ASSIGN 2023-02-08 02:54:25,055 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:54:25,204 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:54:25,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2a409a2959c68e7714a42d395c76489c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865205"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824865205"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824865205"}]},"ts":"1675824865205"} 2023-02-08 02:54:25,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 2a409a2959c68e7714a42d395c76489c, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:25,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b1f619421cec5a6184b63bb1631f4a72, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,209 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865209"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824865209"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824865209"}]},"ts":"1675824865209"} 2023-02-08 02:54:25,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure b1f619421cec5a6184b63bb1631f4a72, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:25,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b1f619421cec5a6184b63bb1631f4a72, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.', STARTKEY => '', ENDKEY => '60'} 2023-02-08 02:54:25,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:25,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:25,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,364 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,365 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1 2023-02-08 02:54:25,365 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1 2023-02-08 02:54:25,365 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1f619421cec5a6184b63bb1631f4a72 columnFamilyName cf1 2023-02-08 02:54:25,375 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-bottom 2023-02-08 02:54:25,379 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-bottom 2023-02-08 02:54:25,383 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,384 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-bottom 2023-02-08 02:54:25,388 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-bottom 2023-02-08 02:54:25,392 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e 2023-02-08 02:54:25,396 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f 2023-02-08 02:54:25,399 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a 2023-02-08 02:54:25,399 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(310): Store=b1f619421cec5a6184b63bb1631f4a72/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:25,399 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,400 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2 2023-02-08 02:54:25,400 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2 2023-02-08 02:54:25,401 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b1f619421cec5a6184b63bb1631f4a72 columnFamilyName cf2 2023-02-08 02:54:25,415 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-bottom 2023-02-08 02:54:25,420 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-bottom 2023-02-08 02:54:25,425 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-bottom 2023-02-08 02:54:25,428 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,429 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-bottom 2023-02-08 02:54:25,432 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a 2023-02-08 02:54:25,436 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5 2023-02-08 02:54:25,439 DEBUG [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60 2023-02-08 02:54:25,439 INFO [StoreOpener-b1f619421cec5a6184b63bb1631f4a72-1] regionserver.HStore(310): Store=b1f619421cec5a6184b63bb1631f4a72/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:25,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,441 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:25,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:25,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened b1f619421cec5a6184b63bb1631f4a72; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63426957, jitterRate=-0.05486468970775604}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:25,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for b1f619421cec5a6184b63bb1631f4a72: Running coprocessor pre-open hook at 1675824865362Writing region info on filesystem at 1675824865363 (+1 ms)Initializing all the Stores at 1675824865363Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824865363Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824865364 (+1 ms)Cleaning up temporary data from old regions at 1675824865441 (+77 ms)Cleaning up detritus from prior splits at 1675824865441Running coprocessor post-open hooks at 1675824865444 (+3 ms)Region opened successfully at 1675824865444 2023-02-08 02:54:25,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72., pid=78, masterSystemTime=1675824865359 2023-02-08 02:54:25,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,446 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:25,446 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:25,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,446 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): b1f619421cec5a6184b63bb1631f4a72/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:25,446 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): b1f619421cec5a6184b63bb1631f4a72/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b1f619421cec5a6184b63bb1631f4a72/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b1f619421cec5a6184b63bb1631f4a72/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp, totalSize=35.0 K 2023-02-08 02:54:25,446 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp, totalSize=35.0 K 2023-02-08 02:54:25,447 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824861609 2023-02-08 02:54:25,447 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824861609 2023-02-08 02:54:25,448 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824861882 2023-02-08 02:54:25,448 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824861882 2023-02-08 02:54:25,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,448 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824862127 2023-02-08 02:54:25,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,448 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824862127 2023-02-08 02:54:25,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2a409a2959c68e7714a42d395c76489c, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.', STARTKEY => '60', ENDKEY => ''} 2023-02-08 02:54:25,448 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824862390 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824862390 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824862663 2023-02-08 02:54:25,449 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=b1f619421cec5a6184b63bb1631f4a72, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824862663 2023-02-08 02:54:25,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824863076 2023-02-08 02:54:25,449 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865449"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824865449"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824865449"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824865449"}]},"ts":"1675824865449"} 2023-02-08 02:54:25,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:25,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824863076 2023-02-08 02:54:25,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,449 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824863410 2023-02-08 02:54:25,450 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824863410 2023-02-08 02:54:25,452 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,453 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1 2023-02-08 02:54:25,453 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1 2023-02-08 02:54:25,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-08 02:54:25,453 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a409a2959c68e7714a42d395c76489c columnFamilyName cf1 2023-02-08 02:54:25,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure b1f619421cec5a6184b63bb1631f4a72, server=jenkins-hbase12.apache.org,37605,1675824833879 in 240 msec 2023-02-08 02:54:25,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, ASSIGN in 402 msec 2023-02-08 02:54:25,465 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): b1f619421cec5a6184b63bb1631f4a72#cf1#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,466 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): b1f619421cec5a6184b63bb1631f4a72#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,469 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-top 2023-02-08 02:54:25,473 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-top 2023-02-08 02:54:25,476 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,476 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-top 2023-02-08 02:54:25,480 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-top 2023-02-08 02:54:25,482 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca 2023-02-08 02:54:25,483 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:25,483 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:25,485 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp/cf2/1effa4b96dbd42f1b6e672d2650791e7 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/1effa4b96dbd42f1b6e672d2650791e7 2023-02-08 02:54:25,489 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b1f619421cec5a6184b63bb1631f4a72/cf1 of b1f619421cec5a6184b63bb1631f4a72 into f2c3bfbec2a049ba9f9c668c0d5765ca(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,489 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b1f619421cec5a6184b63bb1631f4a72: 2023-02-08 02:54:25,489 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72., storeName=b1f619421cec5a6184b63bb1631f4a72/cf1, priority=9, startTime=1675824865445; duration=0sec 2023-02-08 02:54:25,489 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,490 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:25,490 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:25,493 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:25,493 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:25,494 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(310): Store=2a409a2959c68e7714a42d395c76489c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:25,494 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,495 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2 2023-02-08 02:54:25,495 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b1f619421cec5a6184b63bb1631f4a72/cf2 of b1f619421cec5a6184b63bb1631f4a72 into 1effa4b96dbd42f1b6e672d2650791e7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,495 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2 2023-02-08 02:54:25,495 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b1f619421cec5a6184b63bb1631f4a72: 2023-02-08 02:54:25,495 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72., storeName=b1f619421cec5a6184b63bb1631f4a72/cf2, priority=9, startTime=1675824865446; duration=0sec 2023-02-08 02:54:25,495 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,495 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a409a2959c68e7714a42d395c76489c columnFamilyName cf2 2023-02-08 02:54:25,505 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-top 2023-02-08 02:54:25,509 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-top 2023-02-08 02:54:25,513 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-top 2023-02-08 02:54:25,516 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:25,516 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-top 2023-02-08 02:54:25,519 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:25,520 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:25,523 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:25,524 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:25,527 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:25,527 DEBUG [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:25,527 INFO [StoreOpener-2a409a2959c68e7714a42d395c76489c-1] regionserver.HStore(310): Store=2a409a2959c68e7714a42d395c76489c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:25,528 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:54:25,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:25,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 2a409a2959c68e7714a42d395c76489c; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74600644, jitterRate=0.111636221408844}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:54:25,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 2a409a2959c68e7714a42d395c76489c: Running coprocessor pre-open hook at 1675824865449Writing region info on filesystem at 1675824865449Initializing all the Stores at 1675824865450 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824865451 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824865451Cleaning up temporary data from old regions at 1675824865530 (+79 ms)Cleaning up detritus from prior splits at 1675824865530Running coprocessor post-open hooks at 1675824865532 (+2 ms)Region opened successfully at 1675824865532 2023-02-08 02:54:25,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c., pid=77, masterSystemTime=1675824865359 2023-02-08 02:54:25,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,534 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:25,534 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:54:25,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,534 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): 2a409a2959c68e7714a42d395c76489c/cf1 is initiating minor compaction (all files) 2023-02-08 02:54:25,534 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): 2a409a2959c68e7714a42d395c76489c/cf2 is initiating minor compaction (all files) 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2a409a2959c68e7714a42d395c76489c/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2a409a2959c68e7714a42d395c76489c/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp, totalSize=34.8 K 2023-02-08 02:54:25,534 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp, totalSize=34.8 K 2023-02-08 02:54:25,535 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824861609 2023-02-08 02:54:25,535 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824861609 2023-02-08 02:54:25,535 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824861882 2023-02-08 02:54:25,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824862127 2023-02-08 02:54:25,535 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824861882 2023-02-08 02:54:25,536 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=2a409a2959c68e7714a42d395c76489c, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675824863410 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824862127 2023-02-08 02:54:25,536 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865536"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824865536"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824865536"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824865536"}]},"ts":"1675824865536"} 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675824863410 2023-02-08 02:54:25,536 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,537 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,537 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,537 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,537 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-08 02:54:25,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 2a409a2959c68e7714a42d395c76489c, server=jenkins-hbase12.apache.org,37605,1675824833879 in 331 msec 2023-02-08 02:54:25,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-08 02:54:25,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, ASSIGN in 488 msec 2023-02-08 02:54:25,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c in 1.3730 sec 2023-02-08 02:54:25,554 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 2a409a2959c68e7714a42d395c76489c#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,556 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2a409a2959c68e7714a42d395c76489c#cf1#compaction#75 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,579 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ca6c60409d0c440f97fef76ba3df05e1: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,579 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bae97fe8cf5b4440bbea778987dac966: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,579 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp/cf2/ca6c60409d0c440f97fef76ba3df05e1 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ca6c60409d0c440f97fef76ba3df05e1 2023-02-08 02:54:25,580 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp/cf1/bae97fe8cf5b4440bbea778987dac966 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/bae97fe8cf5b4440bbea778987dac966 2023-02-08 02:54:25,583 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for bae97fe8cf5b4440bbea778987dac966: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,583 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ca6c60409d0c440f97fef76ba3df05e1: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,584 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2a409a2959c68e7714a42d395c76489c/cf2 of 2a409a2959c68e7714a42d395c76489c into ca6c60409d0c440f97fef76ba3df05e1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,584 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2a409a2959c68e7714a42d395c76489c/cf1 of 2a409a2959c68e7714a42d395c76489c into bae97fe8cf5b4440bbea778987dac966(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,584 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2a409a2959c68e7714a42d395c76489c: 2023-02-08 02:54:25,584 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2a409a2959c68e7714a42d395c76489c: 2023-02-08 02:54:25,584 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c., storeName=2a409a2959c68e7714a42d395c76489c/cf2, priority=9, startTime=1675824865534; duration=0sec 2023-02-08 02:54:25,584 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c., storeName=2a409a2959c68e7714a42d395c76489c/cf1, priority=9, startTime=1675824865533; duration=0sec 2023-02-08 02:54:25,584 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,584 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,686 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 02:54:25,689 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 136.243.104.168:60332, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 02:54:25,689 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,690 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 02:54:25,690 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HStore(1912): b1f619421cec5a6184b63bb1631f4a72/cf1 is initiating major compaction (all files) 2023-02-08 02:54:25,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@52f5dc34; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,691 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b1f619421cec5a6184b63bb1631f4a72/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 02:54:25,691 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp, totalSize=6.8 K 2023-02-08 02:54:25,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HStore(1912): b1f619421cec5a6184b63bb1631f4a72/cf2 is initiating major compaction (all files) 2023-02-08 02:54:25,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39f71750; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:25,691 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b1f619421cec5a6184b63bb1631f4a72/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:25,691 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/1effa4b96dbd42f1b6e672d2650791e7] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp, totalSize=6.8 K 2023-02-08 02:54:25,691 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 1effa4b96dbd42f1b6e672d2650791e7, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824861609 2023-02-08 02:54:25,691 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting f2c3bfbec2a049ba9f9c668c0d5765ca, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824861609 2023-02-08 02:54:25,693 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.HStore(1912): 2a409a2959c68e7714a42d395c76489c/cf1 is initiating major compaction (all files) 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@596531e6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.HStore(1912): 2a409a2959c68e7714a42d395c76489c/cf2 is initiating major compaction (all files) 2023-02-08 02:54:25,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f90b43d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 02:54:25,695 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 02:54:25,695 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): b1f619421cec5a6184b63bb1631f4a72#cf1#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,696 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): b1f619421cec5a6184b63bb1631f4a72#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,707 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. is not mergeable because it has references 2023-02-08 02:54:25,708 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. is not mergeable because it has references 2023-02-08 02:54:25,718 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp/cf1/2ba551ea7f9e40e4947241d3dbca4c5f as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2ba551ea7f9e40e4947241d3dbca4c5f 2023-02-08 02:54:25,720 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/.tmp/cf2/d50d8d83743c4fea8e196096bcd3b5c7 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/d50d8d83743c4fea8e196096bcd3b5c7 2023-02-08 02:54:25,724 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b1f619421cec5a6184b63bb1631f4a72/cf1 of b1f619421cec5a6184b63bb1631f4a72 into 2ba551ea7f9e40e4947241d3dbca4c5f(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,724 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b1f619421cec5a6184b63bb1631f4a72: 2023-02-08 02:54:25,725 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675824865690; duration=0sec 2023-02-08 02:54:25,725 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in b1f619421cec5a6184b63bb1631f4a72/cf2 of b1f619421cec5a6184b63bb1631f4a72 into d50d8d83743c4fea8e196096bcd3b5c7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,725 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 02:54:25,725 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b1f619421cec5a6184b63bb1631f4a72: 2023-02-08 02:54:25,726 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2a409a2959c68e7714a42d395c76489c/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,726 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675824865691; duration=0sec 2023-02-08 02:54:25,726 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/bae97fe8cf5b4440bbea778987dac966] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp, totalSize=5.0 K 2023-02-08 02:54:25,726 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:54:25,726 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2a409a2959c68e7714a42d395c76489c/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:25,726 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ca6c60409d0c440f97fef76ba3df05e1] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp, totalSize=5.0 K 2023-02-08 02:54:25,726 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting bae97fe8cf5b4440bbea778987dac966, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,726 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting ca6c60409d0c440f97fef76ba3df05e1, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-08 02:54:25,729 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2a409a2959c68e7714a42d395c76489c#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,730 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 2a409a2959c68e7714a42d395c76489c#cf2#compaction#79 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:54:25,747 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ab07edd8a5ed4cc4aeda39dff81324a7: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,748 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp/cf2/ab07edd8a5ed4cc4aeda39dff81324a7 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ab07edd8a5ed4cc4aeda39dff81324a7 2023-02-08 02:54:25,748 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a6d86ce284224b27baf7bfc70330f45d: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,749 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/.tmp/cf1/a6d86ce284224b27baf7bfc70330f45d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/a6d86ce284224b27baf7bfc70330f45d 2023-02-08 02:54:25,752 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ab07edd8a5ed4cc4aeda39dff81324a7: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,753 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2a409a2959c68e7714a42d395c76489c/cf2 of 2a409a2959c68e7714a42d395c76489c into ab07edd8a5ed4cc4aeda39dff81324a7(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,753 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2a409a2959c68e7714a42d395c76489c: 2023-02-08 02:54:25,753 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675824865694; duration=0sec 2023-02-08 02:54:25,753 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,754 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for a6d86ce284224b27baf7bfc70330f45d: NONE, but ROW specified in column family configuration 2023-02-08 02:54:25,755 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2a409a2959c68e7714a42d395c76489c/cf1 of 2a409a2959c68e7714a42d395c76489c into a6d86ce284224b27baf7bfc70330f45d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:54:25,755 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2a409a2959c68e7714a42d395c76489c: 2023-02-08 02:54:25,755 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675824865694; duration=0sec 2023-02-08 02:54:25,755 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:54:25,817 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. is not mergeable because it has references 2023-02-08 02:54:25,817 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37605] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. is not mergeable because it has references 2023-02-08 02:54:25,824 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-08 02:54:25,826 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:25,826 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:25,826 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:25,827 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:25,827 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:25,828 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:25,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:25,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:25,831 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:25,831 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:25,832 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824865832"}]},"ts":"1675824865832"} 2023-02-08 02:54:25,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 02:54:25,833 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-08 02:54:25,886 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-08 02:54:25,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, UNASSIGN}] 2023-02-08 02:54:25,888 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, UNASSIGN 2023-02-08 02:54:25,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2a409a2959c68e7714a42d395c76489c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865888"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824865888"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824865888"}]},"ts":"1675824865888"} 2023-02-08 02:54:25,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 2a409a2959c68e7714a42d395c76489c, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:25,892 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, UNASSIGN 2023-02-08 02:54:25,892 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b1f619421cec5a6184b63bb1631f4a72, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:25,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824865892"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824865892"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824865892"}]},"ts":"1675824865892"} 2023-02-08 02:54:25,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure b1f619421cec5a6184b63bb1631f4a72, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:25,932 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:25,932 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:25,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 02:54:26,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:26,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 2a409a2959c68e7714a42d395c76489c, disabling compactions & flushes 2023-02-08 02:54:26,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:26,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:26,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. after waiting 0 ms 2023-02-08 02:54:26,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:26,043 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/bae97fe8cf5b4440bbea778987dac966, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b] to archive 2023-02-08 02:54:26,044 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:26,046 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,047 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,048 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,049 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,050 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:26,051 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:26,052 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/bae97fe8cf5b4440bbea778987dac966 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/bae97fe8cf5b4440bbea778987dac966 2023-02-08 02:54:26,053 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:26,054 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ca6c60409d0c440f97fef76ba3df05e1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9] to archive 2023-02-08 02:54:26,055 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:26,056 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,057 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,058 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,060 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,061 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:26,062 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:26,063 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ca6c60409d0c440f97fef76ba3df05e1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ca6c60409d0c440f97fef76ba3df05e1 2023-02-08 02:54:26,064 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:26,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-08 02:54:26,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:26,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c. 2023-02-08 02:54:26,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 2a409a2959c68e7714a42d395c76489c: Waiting for close lock at 1675824866042Running coprocessor pre-close hooks at 1675824866042Disabling compacts and flushes for region at 1675824866042Disabling writes for close at 1675824866042Writing region close event to WAL at 1675824866065 (+23 ms)Running coprocessor post-close hooks at 1675824866068 (+3 ms)Closed at 1675824866068 2023-02-08 02:54:26,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:26,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:26,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing b1f619421cec5a6184b63bb1631f4a72, disabling compactions & flushes 2023-02-08 02:54:26,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:26,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:26,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. after waiting 0 ms 2023-02-08 02:54:26,070 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=2a409a2959c68e7714a42d395c76489c, regionState=CLOSED 2023-02-08 02:54:26,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:26,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824866070"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824866070"}]},"ts":"1675824866070"} 2023-02-08 02:54:26,072 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c-bottom] to archive 2023-02-08 02:54:26,072 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:26,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-08 02:54:26,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 2a409a2959c68e7714a42d395c76489c, server=jenkins-hbase12.apache.org,37605,1675824833879 in 182 msec 2023-02-08 02:54:26,074 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2904fbb41ea54a799fb4538e24ddd0ea.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2a409a2959c68e7714a42d395c76489c, UNASSIGN in 187 msec 2023-02-08 02:54:26,075 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7503e7228a3c4afe92c95cd6da8b6089.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,075 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/9efc60edeee5492bbb97be409fe8cbe8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,076 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-a50c9e5096fe4444a270d651d46fb35a 2023-02-08 02:54:26,077 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-850f59ce6e014ba7a92508246a5d346f 2023-02-08 02:54:26,078 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-2086baacc7a348149441de014ac9a43e 2023-02-08 02:54:26,079 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/f2c3bfbec2a049ba9f9c668c0d5765ca 2023-02-08 02:54:26,080 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/7e3e270c82174471803086e830f32b0c.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,081 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/1effa4b96dbd42f1b6e672d2650791e7, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8-bottom] to archive 2023-02-08 02:54:26,082 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:54:26,083 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/75225cc3506a43a18222d00b51096438.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,084 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/170989de44814a738f48f47ba3dd7c86.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,085 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/85fca2aa40a64420a233e048f3e9330d.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,086 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-dd1e7cd5c11a4648936e9073a5759e60 2023-02-08 02:54:26,086 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-567e05b7199b44039a115ae10b522ca5 2023-02-08 02:54:26,087 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/testSplitTableRegionDeletedRowsDaughter=833776ad2367c20696c3f51f3ecac4a3-38521ca7fcfd4e60961875241269f48a 2023-02-08 02:54:26,088 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/1effa4b96dbd42f1b6e672d2650791e7 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/1effa4b96dbd42f1b6e672d2650791e7 2023-02-08 02:54:26,089 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/bd203221148e46fabfe184e82202baf8.833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-08 02:54:26,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:26,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72. 2023-02-08 02:54:26,093 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for b1f619421cec5a6184b63bb1631f4a72: Waiting for close lock at 1675824866070Running coprocessor pre-close hooks at 1675824866070Disabling compacts and flushes for region at 1675824866070Disabling writes for close at 1675824866070Writing region close event to WAL at 1675824866090 (+20 ms)Running coprocessor post-close hooks at 1675824866093 (+3 ms)Closed at 1675824866093 2023-02-08 02:54:26,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:26,095 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=b1f619421cec5a6184b63bb1631f4a72, regionState=CLOSED 2023-02-08 02:54:26,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675824866095"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824866095"}]},"ts":"1675824866095"} 2023-02-08 02:54:26,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-08 02:54:26,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure b1f619421cec5a6184b63bb1631f4a72, server=jenkins-hbase12.apache.org,37605,1675824833879 in 202 msec 2023-02-08 02:54:26,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-08 02:54:26,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=b1f619421cec5a6184b63bb1631f4a72, UNASSIGN in 210 msec 2023-02-08 02:54:26,099 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824866099"}]},"ts":"1675824866099"} 2023-02-08 02:54:26,100 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-08 02:54:26,107 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-08 02:54:26,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 280 msec 2023-02-08 02:54:26,133 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,134 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 02:54:26,135 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-08 02:54:26,135 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,135 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,136 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,138 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,139 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,139 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,139 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 02:54:26,143 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,143 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:26,143 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:26,145 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/recovered.edits] 2023-02-08 02:54:26,145 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/recovered.edits] 2023-02-08 02:54:26,145 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/recovered.edits] 2023-02-08 02:54:26,147 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2ba551ea7f9e40e4947241d3dbca4c5f to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf1/2ba551ea7f9e40e4947241d3dbca4c5f 2023-02-08 02:54:26,148 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0b7bc862d7724752aea437fa6606c4dd 2023-02-08 02:54:26,148 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/a6d86ce284224b27baf7bfc70330f45d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf1/a6d86ce284224b27baf7bfc70330f45d 2023-02-08 02:54:26,149 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/0f448898212947b2a8cd1418ea869277 2023-02-08 02:54:26,149 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ab07edd8a5ed4cc4aeda39dff81324a7 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/cf2/ab07edd8a5ed4cc4aeda39dff81324a7 2023-02-08 02:54:26,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2086baacc7a348149441de014ac9a43e 2023-02-08 02:54:26,150 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/d50d8d83743c4fea8e196096bcd3b5c7 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/cf2/d50d8d83743c4fea8e196096bcd3b5c7 2023-02-08 02:54:26,151 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/2904fbb41ea54a799fb4538e24ddd0ea 2023-02-08 02:54:26,152 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/536a6952ab494b32a1cf57f68c09645b 2023-02-08 02:54:26,152 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/recovered.edits/113.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c/recovered.edits/113.seqid 2023-02-08 02:54:26,152 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/recovered.edits/113.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72/recovered.edits/113.seqid 2023-02-08 02:54:26,153 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2a409a2959c68e7714a42d395c76489c 2023-02-08 02:54:26,153 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/b1f619421cec5a6184b63bb1631f4a72 2023-02-08 02:54:26,153 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7503e7228a3c4afe92c95cd6da8b6089 2023-02-08 02:54:26,154 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/7e3e270c82174471803086e830f32b0c 2023-02-08 02:54:26,155 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/850f59ce6e014ba7a92508246a5d346f 2023-02-08 02:54:26,156 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/9efc60edeee5492bbb97be409fe8cbe8 2023-02-08 02:54:26,157 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf1/a50c9e5096fe4444a270d651d46fb35a 2023-02-08 02:54:26,160 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/00c3af6c67344a51b3da13bf09842ea4 2023-02-08 02:54:26,161 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/170989de44814a738f48f47ba3dd7c86 2023-02-08 02:54:26,162 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/38521ca7fcfd4e60961875241269f48a 2023-02-08 02:54:26,163 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/419b49e58f894c0084eb863dc918da8b 2023-02-08 02:54:26,164 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/4f79a8f8588e4507a0adb58078c961b9 2023-02-08 02:54:26,165 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/567e05b7199b44039a115ae10b522ca5 2023-02-08 02:54:26,166 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/75225cc3506a43a18222d00b51096438 2023-02-08 02:54:26,167 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/85fca2aa40a64420a233e048f3e9330d 2023-02-08 02:54:26,168 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/bd203221148e46fabfe184e82202baf8 2023-02-08 02:54:26,169 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/cf2/dd1e7cd5c11a4648936e9073a5759e60 2023-02-08 02:54:26,172 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/recovered.edits/106.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3/recovered.edits/106.seqid 2023-02-08 02:54:26,172 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/833776ad2367c20696c3f51f3ecac4a3 2023-02-08 02:54:26,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-08 02:54:26,174 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,176 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-08 02:54:26,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-08 02:54:26,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,179 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-08 02:54:26,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824866179"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:26,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824866179"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:26,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824866179"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:26,182 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:54:26,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 833776ad2367c20696c3f51f3ecac4a3, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675824861105.833776ad2367c20696c3f51f3ecac4a3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => b1f619421cec5a6184b63bb1631f4a72, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675824864170.b1f619421cec5a6184b63bb1631f4a72.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 2a409a2959c68e7714a42d395c76489c, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675824864170.2a409a2959c68e7714a42d395c76489c.', STARTKEY => '60', ENDKEY => ''}] 2023-02-08 02:54:26,182 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-08 02:54:26,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824866182"}]},"ts":"9223372036854775807"} 2023-02-08 02:54:26,184 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-08 02:54:26,192 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:54:26,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 56 msec 2023-02-08 02:54:26,240 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,240 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-08 02:54:26,241 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-08 02:54:26,255 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=413 (was 408) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:52252 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48608 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44112 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44070 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48350 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:40300 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46092 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:40222 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:48640 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:52260 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44376 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:52284 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:40232 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x187a2033-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x196478f2-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:46364 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44096 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-261156481_15 at /127.0.0.1:44054 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=853 (was 858), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=279 (was 277) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=3615 (was 3627) 2023-02-08 02:54:26,268 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=413, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=173, AvailableMemoryMB=3615 2023-02-08 02:54:26,268 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:54:26,269 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,269 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:54:26,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:54:26,285 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:54:26,286 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:54:26,287 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:54:26,289 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,289 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 empty. 2023-02-08 02:54:26,290 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,290 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 02:54:26,299 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:26,300 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5dfcb5ec6b56adf0449fc6c004177ac9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 5dfcb5ec6b56adf0449fc6c004177ac9, disabling compactions & flushes 2023-02-08 02:54:26,307 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. after waiting 0 ms 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,307 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,307 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 5dfcb5ec6b56adf0449fc6c004177ac9: Waiting for close lock at 1675824866307Disabling compacts and flushes for region at 1675824866307Disabling writes for close at 1675824866307Writing region close event to WAL at 1675824866307Closed at 1675824866307 2023-02-08 02:54:26,309 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:54:26,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824866310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824866310"}]},"ts":"1675824866310"} 2023-02-08 02:54:26,311 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:54:26,312 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:54:26,312 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824866312"}]},"ts":"1675824866312"} 2023-02-08 02:54:26,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-08 02:54:26,378 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:26,379 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:26,379 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:26,379 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:26,379 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:54:26,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, ASSIGN}] 2023-02-08 02:54:26,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, ASSIGN 2023-02-08 02:54:26,381 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,46783,1675824833788; forceNewPlan=false, retain=false 2023-02-08 02:54:26,531 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:26,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:26,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824866533"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824866533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824866533"}]},"ts":"1675824866533"} 2023-02-08 02:54:26,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:26,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5dfcb5ec6b56adf0449fc6c004177ac9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:26,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:26,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:26,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,691 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,692 DEBUG [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:54:26,692 DEBUG [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:54:26,693 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5dfcb5ec6b56adf0449fc6c004177ac9 columnFamilyName cf1 2023-02-08 02:54:26,693 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] regionserver.HStore(310): Store=5dfcb5ec6b56adf0449fc6c004177ac9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:26,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,694 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:54:26,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 5dfcb5ec6b56adf0449fc6c004177ac9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58817517, jitterRate=-0.12355069816112518}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 02:54:26,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 5dfcb5ec6b56adf0449fc6c004177ac9: Running coprocessor pre-open hook at 1675824866690Writing region info on filesystem at 1675824866690Initializing all the Stores at 1675824866691 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824866691Cleaning up temporary data from old regions at 1675824866695 (+4 ms)Cleaning up detritus from prior splits at 1675824866695Running coprocessor post-open hooks at 1675824866700 (+5 ms)Region opened successfully at 1675824866700 2023-02-08 02:54:26,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9., pid=87, masterSystemTime=1675824866687 2023-02-08 02:54:26,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,703 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:26,703 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824866703"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824866703"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824866703"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824866703"}]},"ts":"1675824866703"} 2023-02-08 02:54:26,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-08 02:54:26,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788 in 169 msec 2023-02-08 02:54:26,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-08 02:54:26,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, ASSIGN in 327 msec 2023-02-08 02:54:26,708 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:54:26,709 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824866708"}]},"ts":"1675824866708"} 2023-02-08 02:54:26,710 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-08 02:54:26,761 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:54:26,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 477 msec 2023-02-08 02:54:26,786 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,786 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,788 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:54:26,788 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:54:26,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$12(2441): Client=jenkins//136.243.104.168 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 02:54:26,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:54:26,814 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 02:54:26,815 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-08 02:54:26,815 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-08 02:54:26,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-08 02:54:26,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, REOPEN/MOVE}] 2023-02-08 02:54:26,829 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, REOPEN/MOVE 2023-02-08 02:54:26,830 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:26,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824866830"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824866830"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824866830"}]},"ts":"1675824866830"} 2023-02-08 02:54:26,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:26,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 5dfcb5ec6b56adf0449fc6c004177ac9, disabling compactions & flushes 2023-02-08 02:54:26,984 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. after waiting 0 ms 2023-02-08 02:54:26,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-08 02:54:26,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:54:26,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:26,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 5dfcb5ec6b56adf0449fc6c004177ac9: Waiting for close lock at 1675824866983Running coprocessor pre-close hooks at 1675824866983Disabling compacts and flushes for region at 1675824866983Disabling writes for close at 1675824866984 (+1 ms)Writing region close event to WAL at 1675824866985 (+1 ms)Running coprocessor post-close hooks at 1675824866988 (+3 ms)Closed at 1675824866988 2023-02-08 02:54:26,988 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 5dfcb5ec6b56adf0449fc6c004177ac9 to self. 2023-02-08 02:54:26,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:26,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=CLOSED 2023-02-08 02:54:26,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824866990"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824866990"}]},"ts":"1675824866990"} 2023-02-08 02:54:26,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-08 02:54:26,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788 in 160 msec 2023-02-08 02:54:26,993 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase12.apache.org,46783,1675824833788; forceNewPlan=false, retain=true 2023-02-08 02:54:27,144 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:54:27,144 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:27,144 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824867144"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824867144"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824867144"}]},"ts":"1675824867144"} 2023-02-08 02:54:27,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:54:27,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:27,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5dfcb5ec6b56adf0449fc6c004177ac9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:54:27,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:27,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:27,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,302 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,303 DEBUG [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:54:27,303 DEBUG [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:54:27,303 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5dfcb5ec6b56adf0449fc6c004177ac9 columnFamilyName cf1 2023-02-08 02:54:27,304 INFO [StoreOpener-5dfcb5ec6b56adf0449fc6c004177ac9-1] regionserver.HStore(310): Store=5dfcb5ec6b56adf0449fc6c004177ac9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:54:27,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:54:27,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 5dfcb5ec6b56adf0449fc6c004177ac9; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66273178, jitterRate=-0.012452691793441772}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 02:54:27,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 5dfcb5ec6b56adf0449fc6c004177ac9: Running coprocessor pre-open hook at 1675824867301Writing region info on filesystem at 1675824867301Initializing all the Stores at 1675824867302 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824867302Cleaning up temporary data from old regions at 1675824867305 (+3 ms)Cleaning up detritus from prior splits at 1675824867306 (+1 ms)Running coprocessor post-open hooks at 1675824867308 (+2 ms)Region opened successfully at 1675824867308 2023-02-08 02:54:27,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9., pid=92, masterSystemTime=1675824867297 2023-02-08 02:54:27,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:27,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:54:27,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:54:27,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824867311"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824867311"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824867311"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824867311"}]},"ts":"1675824867311"} 2023-02-08 02:54:27,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-08 02:54:27,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788 in 166 msec 2023-02-08 02:54:27,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-08 02:54:27,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, REOPEN/MOVE in 485 msec 2023-02-08 02:54:27,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-08 02:54:27,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-02-08 02:54:27,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, ASSIGN}] 2023-02-08 02:54:27,320 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, ASSIGN 2023-02-08 02:54:27,321 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 02:54:27,471 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:54:27,473 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:54:27,473 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:54:27,473 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:54:27,473 DEBUG [jenkins-hbase12:39013] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-08 02:54:27,474 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=39b49a9b2047bfbdfdf109b4bad2ab6d, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:54:27,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824867474"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675824867474"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675824867474"}]},"ts":"1675824867474"} 2023-02-08 02:54:27,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 39b49a9b2047bfbdfdf109b4bad2ab6d, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:54:27,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:54:27,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 39b49a9b2047bfbdfdf109b4bad2ab6d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-08 02:54:27,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:54:27,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:54:27,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:54:27,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:54:27,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:54:27,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:27,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:27,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:27,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:28,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:29,874 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:54:29,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:30,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,523 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-08 02:54:31,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:31,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:32,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:33,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:34,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:35,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:36,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:37,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:38,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:39,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:40,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:41,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:42,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:43,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:44,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:45,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:46,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:47,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:48,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:49,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:50,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:51,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:52,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:53,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:54,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,402 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 02:54:55,431 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 02:54:55,466 DEBUG [regionserver/jenkins-hbase12:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-08 02:54:55,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:55,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:56,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:57,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:58,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:54:59,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:00,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:01,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:02,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:03,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:04,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:05,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:06,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:07,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:08,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:09,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:10,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:11,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:12,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:13,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:14,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:15,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:16,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:17,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:18,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:19,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:20,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:21,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:22,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:23,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:24,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:25,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:26,909 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:26,909 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:26,915 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:26,915 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:26,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 02:55:26,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,016 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:27,016 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:27,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 02:55:27,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,217 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:27,218 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:27,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 02:55:27,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,519 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:27,519 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:27,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 02:55:27,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:27,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,021 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:28,022 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:28,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 02:55:28,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:28,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:29,026 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:29,027 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:29,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:29,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:29,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:29,031 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 5dfcb5ec6b56adf0449fc6c004177ac9, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:29,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 exec-time=7 msec 2023-02-08 02:55:29,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:29,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:29,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-08 02:55:29,373 INFO [StoreOpener-39b49a9b2047bfbdfdf109b4bad2ab6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:55:29,378 DEBUG [StoreOpener-39b49a9b2047bfbdfdf109b4bad2ab6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:55:29,378 DEBUG [StoreOpener-39b49a9b2047bfbdfdf109b4bad2ab6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1 2023-02-08 02:55:29,379 INFO [StoreOpener-39b49a9b2047bfbdfdf109b4bad2ab6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 39b49a9b2047bfbdfdf109b4bad2ab6d columnFamilyName cf1 2023-02-08 02:55:29,380 INFO [StoreOpener-39b49a9b2047bfbdfdf109b4bad2ab6d-1] regionserver.HStore(310): Store=39b49a9b2047bfbdfdf109b4bad2ab6d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:29,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 39b49a9b2047bfbdfdf109b4bad2ab6d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71695817, jitterRate=0.06835092604160309}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 02:55:29,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 39b49a9b2047bfbdfdf109b4bad2ab6d: Running coprocessor pre-open hook at 1675824867633Initializing all the Stores at 1675824929373 (+61740 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824929373Running coprocessor post-open hooks at 1675824929382 (+9 ms)Region opened successfully at 1675824929382 2023-02-08 02:55:29,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d., pid=94, masterSystemTime=1675824867629 2023-02-08 02:55:29,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:29,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:29,384 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=39b49a9b2047bfbdfdf109b4bad2ab6d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:29,384 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824929384"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1675824929384"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675824929384"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675824929384"}]},"ts":"1675824929384"} 2023-02-08 02:55:29,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-08 02:55:29,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 39b49a9b2047bfbdfdf109b4bad2ab6d, server=jenkins-hbase12.apache.org,37605,1675824833879 in 1 mins, 1.909 sec 2023-02-08 02:55:29,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-08 02:55:29,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, ASSIGN in 1 mins, 2.068 sec 2023-02-08 02:55:29,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.592 sec 2023-02-08 02:55:29,390 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675824866792","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62598,"client":"136.243.104.168:58546","queuetimems":0,"class":"HMaster"} 2023-02-08 02:55:29,391 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 605, skipping over this response of 3 bytes 2023-02-08 02:55:31,278 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up 2023-02-08 02:55:31,280 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:31,280 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,280 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,283 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,283 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,284 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,290 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,290 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 02:55:31,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824931291"}]},"ts":"1675824931291"} 2023-02-08 02:55:31,293 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-08 02:55:31,392 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,392 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 02:55:31,397 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-08 02:55:31,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, UNASSIGN}] 2023-02-08 02:55:31,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, UNASSIGN 2023-02-08 02:55:31,400 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=39b49a9b2047bfbdfdf109b4bad2ab6d, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:31,400 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824931399"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1675824931399"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675824931399"}]},"ts":"1675824931399"} 2023-02-08 02:55:31,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 39b49a9b2047bfbdfdf109b4bad2ab6d, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:31,404 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, UNASSIGN 2023-02-08 02:55:31,405 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:31,405 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824931405"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824931405"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824931405"}]},"ts":"1675824931405"} 2023-02-08 02:55:31,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788}] 2023-02-08 02:55:31,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:55:31,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 39b49a9b2047bfbdfdf109b4bad2ab6d, disabling compactions & flushes 2023-02-08 02:55:31,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:31,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:31,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. after waiting 0 ms 2023-02-08 02:55:31,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:31,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:31,557 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d. 2023-02-08 02:55:31,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 39b49a9b2047bfbdfdf109b4bad2ab6d: Waiting for close lock at 1675824931554Running coprocessor pre-close hooks at 1675824931554Disabling compacts and flushes for region at 1675824931554Disabling writes for close at 1675824931555 (+1 ms)Writing region close event to WAL at 1675824931556 (+1 ms)Running coprocessor post-close hooks at 1675824931556Closed at 1675824931557 (+1 ms) 2023-02-08 02:55:31,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 39b49a9b2047bfbdfdf109b4bad2ab6d 2023-02-08 02:55:31,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:55:31,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 5dfcb5ec6b56adf0449fc6c004177ac9, disabling compactions & flushes 2023-02-08 02:55:31,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:55:31,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:55:31,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. after waiting 0 ms 2023-02-08 02:55:31,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:55:31,561 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=39b49a9b2047bfbdfdf109b4bad2ab6d, regionState=CLOSED 2023-02-08 02:55:31,561 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824931561"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675824931561"}]},"ts":"1675824931561"} 2023-02-08 02:55:31,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-08 02:55:31,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 39b49a9b2047bfbdfdf109b4bad2ab6d, server=jenkins-hbase12.apache.org,37605,1675824833879 in 164 msec 2023-02-08 02:55:31,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-08 02:55:31,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:31,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9. 2023-02-08 02:55:31,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 5dfcb5ec6b56adf0449fc6c004177ac9: Waiting for close lock at 1675824931560Running coprocessor pre-close hooks at 1675824931560Disabling compacts and flushes for region at 1675824931560Disabling writes for close at 1675824931561 (+1 ms)Writing region close event to WAL at 1675824931562 (+1 ms)Running coprocessor post-close hooks at 1675824931569 (+7 ms)Closed at 1675824931569 2023-02-08 02:55:31,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d, UNASSIGN in 170 msec 2023-02-08 02:55:31,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:55:31,571 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=5dfcb5ec6b56adf0449fc6c004177ac9, regionState=CLOSED 2023-02-08 02:55:31,571 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675824931571"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824931571"}]},"ts":"1675824931571"} 2023-02-08 02:55:31,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-08 02:55:31,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 5dfcb5ec6b56adf0449fc6c004177ac9, server=jenkins-hbase12.apache.org,46783,1675824833788 in 166 msec 2023-02-08 02:55:31,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-08 02:55:31,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=5dfcb5ec6b56adf0449fc6c004177ac9, UNASSIGN in 175 msec 2023-02-08 02:55:31,576 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824931576"}]},"ts":"1675824931576"} 2023-02-08 02:55:31,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-08 02:55:31,586 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-08 02:55:31,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 302 msec 2023-02-08 02:55:31,593 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,594 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 02:55:31,595 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-08 02:55:31,595 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,595 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,597 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,597 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,598 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 02:55:31,602 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:55:31,603 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits] 2023-02-08 02:55:31,607 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits/7.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9/recovered.edits/7.seqid 2023-02-08 02:55:31,607 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/5dfcb5ec6b56adf0449fc6c004177ac9 2023-02-08 02:55:31,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-08 02:55:31,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,611 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-08 02:55:31,613 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-08 02:55:31,614 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,614 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-08 02:55:31,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824931614"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:31,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824931614"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:31,616 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-08 02:55:31,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5dfcb5ec6b56adf0449fc6c004177ac9, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284.5dfcb5ec6b56adf0449fc6c004177ac9.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 39b49a9b2047bfbdfdf109b4bad2ab6d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675824866284_0001.39b49a9b2047bfbdfdf109b4bad2ab6d.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-08 02:55:31,616 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-08 02:55:31,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824931616"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:31,618 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-08 02:55:31,629 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:31,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 34 msec 2023-02-08 02:55:31,699 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,699 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-08 02:55:31,701 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-08 02:55:31,715 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 413), OpenFileDescriptor=658 (was 853), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=206 (was 279), ProcessCount=171 (was 173), AvailableMemoryMB=3625 (was 3615) - AvailableMemoryMB LEAK? - 2023-02-08 02:55:31,730 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=206, ProcessCount=171, AvailableMemoryMB=3624 2023-02-08 02:55:31,732 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:31,732 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:31,732 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:31,755 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:55:31,755 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:55:31,757 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:31,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:55:31,760 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:55:31,761 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:31,762 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 empty. 2023-02-08 02:55:31,763 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:31,763 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 02:55:31,774 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-08 02:55:31,775 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 071ac4e2b6107e64d654a922cf0b1516, NAME => 'testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 071ac4e2b6107e64d654a922cf0b1516, disabling compactions & flushes 2023-02-08 02:55:31,781 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. after waiting 0 ms 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:31,781 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:31,781 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 071ac4e2b6107e64d654a922cf0b1516: Waiting for close lock at 1675824931781Disabling compacts and flushes for region at 1675824931781Disabling writes for close at 1675824931781Writing region close event to WAL at 1675824931781Closed at 1675824931781 2023-02-08 02:55:31,783 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:55:31,784 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675824931784"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824931784"}]},"ts":"1675824931784"} 2023-02-08 02:55:31,785 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:55:31,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:55:31,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824931786"}]},"ts":"1675824931786"} 2023-02-08 02:55:31,790 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-08 02:55:31,808 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:55:31,809 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:55:31,809 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:55:31,809 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:55:31,809 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:55:31,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, ASSIGN}] 2023-02-08 02:55:31,810 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, ASSIGN 2023-02-08 02:55:31,810 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:31,961 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:55:31,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=071ac4e2b6107e64d654a922cf0b1516, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:31,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675824931962"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824931962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824931962"}]},"ts":"1675824931962"} 2023-02-08 02:55:31,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 071ac4e2b6107e64d654a922cf0b1516, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:32,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 071ac4e2b6107e64d654a922cf0b1516, NAME => 'testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:55:32,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:32,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:32,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,120 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,121 DEBUG [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1 2023-02-08 02:55:32,121 DEBUG [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1 2023-02-08 02:55:32,121 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 071ac4e2b6107e64d654a922cf0b1516 columnFamilyName cf1 2023-02-08 02:55:32,122 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] regionserver.HStore(310): Store=071ac4e2b6107e64d654a922cf0b1516/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:32,122 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,123 DEBUG [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2 2023-02-08 02:55:32,123 DEBUG [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2 2023-02-08 02:55:32,124 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 071ac4e2b6107e64d654a922cf0b1516 columnFamilyName cf2 2023-02-08 02:55:32,124 INFO [StoreOpener-071ac4e2b6107e64d654a922cf0b1516-1] regionserver.HStore(310): Store=071ac4e2b6107e64d654a922cf0b1516/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:32,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:32,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:32,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:55:32,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 071ac4e2b6107e64d654a922cf0b1516; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61616134, jitterRate=-0.08184805512428284}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:32,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 071ac4e2b6107e64d654a922cf0b1516: Running coprocessor pre-open hook at 1675824932119Writing region info on filesystem at 1675824932119Initializing all the Stores at 1675824932119Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824932119Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824932120 (+1 ms)Cleaning up temporary data from old regions at 1675824932127 (+7 ms)Cleaning up detritus from prior splits at 1675824932127Running coprocessor post-open hooks at 1675824932134 (+7 ms)Region opened successfully at 1675824932134 2023-02-08 02:55:32,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516., pid=104, masterSystemTime=1675824932115 2023-02-08 02:55:32,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=071ac4e2b6107e64d654a922cf0b1516, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,137 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675824932137"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824932137"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824932137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824932137"}]},"ts":"1675824932137"} 2023-02-08 02:55:32,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-08 02:55:32,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 071ac4e2b6107e64d654a922cf0b1516, server=jenkins-hbase12.apache.org,37605,1675824833879 in 176 msec 2023-02-08 02:55:32,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-08 02:55:32,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, ASSIGN in 332 msec 2023-02-08 02:55:32,143 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:55:32,143 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824932143"}]},"ts":"1675824932143"} 2023-02-08 02:55:32,144 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-08 02:55:32,156 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:55:32,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 400 msec 2023-02-08 02:55:32,263 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:32,263 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:32,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:32,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:32,265 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:32,265 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,266 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:32,266 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:32,266 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,266 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,313 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,313 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,313 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,314 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,314 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,315 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,315 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,316 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,316 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,316 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,316 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:32,316 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:32,316 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,316 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:32,317 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:55:32,323 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,328 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/8b029a6c1fc94f7793e6e6bf62750a4f 2023-02-08 02:55:32,334 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,334 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,334 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,334 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,335 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,341 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/88e4b8e85f784baba6d4800872fd91ef 2023-02-08 02:55:32,344 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/8b029a6c1fc94f7793e6e6bf62750a4f as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/8b029a6c1fc94f7793e6e6bf62750a4f 2023-02-08 02:55:32,348 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/8b029a6c1fc94f7793e6e6bf62750a4f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:32,349 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/88e4b8e85f784baba6d4800872fd91ef as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/88e4b8e85f784baba6d4800872fd91ef 2023-02-08 02:55:32,354 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/88e4b8e85f784baba6d4800872fd91ef, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:32,355 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 39ms, sequenceid=6, compaction requested=false 2023-02-08 02:55:32,355 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-08 02:55:32,355 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:32,355 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,355 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:32,355 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:32,355 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,355 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,356 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,365 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,365 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,366 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,366 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:32,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34cb41b1[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:32,368 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:32,368 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,376 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:32,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,376 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,376 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,377 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:32,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:32,377 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,377 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,377 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:32,386 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,386 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,386 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,387 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,386 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,387 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,387 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,387 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,387 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:32,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:32,390 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:32,391 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:32,391 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:32,391 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:32,391 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,397 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,398 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,398 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,399 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,399 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,400 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:32,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:32,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:32,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,408 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:32,409 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,409 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,409 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,409 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,409 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,509 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:32,509 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:32,509 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:32,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:32,529 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:32,529 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:32,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:32,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:32,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:32,532 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:32,532 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,533 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:32,533 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:32,533 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,533 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,571 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,572 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,572 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:32,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,573 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,574 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,575 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,575 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,576 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:32,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:32,577 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,577 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:32,577 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:32,586 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 2023-02-08 02:55:32,597 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,601 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/da542d0d5a454c9395aab6d2526a0292 2023-02-08 02:55:32,604 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 2023-02-08 02:55:32,607 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:32,607 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/da542d0d5a454c9395aab6d2526a0292 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292 2023-02-08 02:55:32,607 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,607 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,607 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,609 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,609 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,611 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:32,612 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 35ms, sequenceid=19, compaction requested=false 2023-02-08 02:55:32,612 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:32,612 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,612 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:32,612 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:32,612 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,612 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,612 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,618 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,618 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,618 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:32,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@683a056d[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:32,620 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:32,620 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,629 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,629 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:32,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,629 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:32,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,650 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,650 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:32,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:32,654 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:32,654 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:32,654 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:32,654 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:32,654 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,661 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,661 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,663 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,663 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,663 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,671 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:32,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:32,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,671 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:32,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,672 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:32,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,772 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:32,772 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:32,772 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:32,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:32,784 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:32,784 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:32,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:32,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:32,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:32,786 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:32,786 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,786 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:32,786 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:32,787 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,787 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:32,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,799 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:32,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:32,799 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,799 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:32,799 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:32,807 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/1c9b05dd26644c1dbe474d692a85ad8e 2023-02-08 02:55:32,807 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,817 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/08fa596c3c814098ae1e49753cb0eb25 2023-02-08 02:55:32,818 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,818 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,818 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,818 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,818 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,820 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/1c9b05dd26644c1dbe474d692a85ad8e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/1c9b05dd26644c1dbe474d692a85ad8e 2023-02-08 02:55:32,824 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/1c9b05dd26644c1dbe474d692a85ad8e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:32,824 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/08fa596c3c814098ae1e49753cb0eb25 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/08fa596c3c814098ae1e49753cb0eb25 2023-02-08 02:55:32,827 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/08fa596c3c814098ae1e49753cb0eb25, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:32,828 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 29ms, sequenceid=32, compaction requested=true 2023-02-08 02:55:32,828 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:32,828 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:32,828 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:32,828 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:32,828 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,828 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:32,828 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:32,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,839 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,839 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,839 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:32,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:32,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44f82842[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:32,841 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:32,841 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,850 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:32,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:32,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:32,860 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,860 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,860 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:32,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,871 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:32,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:32,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:32,874 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:32,874 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:32,874 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:32,874 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:32,874 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:32,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,882 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,881 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,882 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,882 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:32,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,883 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,883 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:32,884 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,919 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:32,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:32,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:32,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:32,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:32,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:32,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:32,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:32,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:32,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,020 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:33,020 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,020 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:33,036 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,036 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:33,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:33,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:33,038 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:33,038 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,038 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:33,038 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:33,039 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,039 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,060 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,060 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,061 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,061 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:33,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,063 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,064 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,064 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,064 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:33,064 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:33,064 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,064 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:33,064 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:33,071 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,077 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/5ce7d326b5e545da8322b4f913cb26b4 2023-02-08 02:55:33,081 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,081 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,081 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,082 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,082 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,092 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/2cab42fc4f8c4687bfc5799df924efee 2023-02-08 02:55:33,097 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/5ce7d326b5e545da8322b4f913cb26b4 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/5ce7d326b5e545da8322b4f913cb26b4 2023-02-08 02:55:33,103 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/5ce7d326b5e545da8322b4f913cb26b4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:33,104 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/2cab42fc4f8c4687bfc5799df924efee as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/2cab42fc4f8c4687bfc5799df924efee 2023-02-08 02:55:33,111 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/2cab42fc4f8c4687bfc5799df924efee, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:33,112 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 48ms, sequenceid=45, compaction requested=true 2023-02-08 02:55:33,112 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:33,112 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,112 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:33,112 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:33,112 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,112 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,113 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,123 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,123 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,124 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,124 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:33,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7eca1b2a[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:33,129 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:33,129 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,134 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,134 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,134 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,135 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,135 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,135 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:33,135 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,135 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:33,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,135 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,136 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,136 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:33,136 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,136 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,136 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:33,145 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,145 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,145 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,145 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,155 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:33,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:33,164 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:33,164 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:33,164 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:33,164 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:33,164 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:33,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,178 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,179 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,180 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,185 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,185 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,186 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:33,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:33,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,197 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:33,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,298 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:33,299 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,299 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:33,322 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,322 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:33,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:33,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:33,325 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:33,325 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,325 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:33,325 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:33,326 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,326 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,334 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,334 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,334 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,334 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,335 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,335 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,337 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,338 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:33,338 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,338 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,338 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:33,344 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,351 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/11a9c11287904e7888a9cb6c1e56ff9e 2023-02-08 02:55:33,355 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,355 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,355 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,355 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,355 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,365 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/f2ddabb057304d55a04e539d95e3999e 2023-02-08 02:55:33,370 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/11a9c11287904e7888a9cb6c1e56ff9e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/11a9c11287904e7888a9cb6c1e56ff9e 2023-02-08 02:55:33,377 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/11a9c11287904e7888a9cb6c1e56ff9e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:33,378 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/f2ddabb057304d55a04e539d95e3999e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/f2ddabb057304d55a04e539d95e3999e 2023-02-08 02:55:33,383 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/f2ddabb057304d55a04e539d95e3999e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:33,384 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 46ms, sequenceid=58, compaction requested=true 2023-02-08 02:55:33,384 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:33,384 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,384 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:33,384 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:33,384 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,384 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,384 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,425 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,425 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,426 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,426 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:33,430 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10621b8b[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:33,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:33,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,489 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:33,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:33,490 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,491 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:33,490 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,491 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,491 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:33,513 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,513 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,513 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,534 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:33,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:33,543 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:33,543 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:33,543 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:33,543 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:33,543 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,558 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,558 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,558 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:33,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:33,566 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,567 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,666 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:33,667 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,667 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:33,688 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,689 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:33,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:33,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-08 02:55:33,691 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-08 02:55:33,691 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,692 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-08 02:55:33,692 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-08 02:55:33,692 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,692 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,702 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,703 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,703 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-08 02:55:33,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,704 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,704 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:33,705 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:33,705 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,705 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. started... 2023-02-08 02:55:33,705 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:33,713 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,715 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/ee5411c74614455ba50f763e5572f8ba 2023-02-08 02:55:33,723 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,723 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,724 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,724 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,724 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,732 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/64efed41889348adbade0658eb1df885 2023-02-08 02:55:33,739 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/ee5411c74614455ba50f763e5572f8ba as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/ee5411c74614455ba50f763e5572f8ba 2023-02-08 02:55:33,745 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/ee5411c74614455ba50f763e5572f8ba, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:33,746 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/64efed41889348adbade0658eb1df885 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/64efed41889348adbade0658eb1df885 2023-02-08 02:55:33,752 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/64efed41889348adbade0658eb1df885, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:33,753 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 48ms, sequenceid=71, compaction requested=true 2023-02-08 02:55:33,753 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:33,753 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:33,754 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:33,754 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:33,754 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,754 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-08 02:55:33,754 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-08 02:55:33,766 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,766 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,766 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,766 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:33,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-08 02:55:33,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@284b6587[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:33,772 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-08 02:55:33,772 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,777 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,777 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,777 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:33,777 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,777 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:33,777 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,777 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-08 02:55:33,778 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,778 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:33,778 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-08 02:55:33,778 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:33,787 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,787 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,787 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,798 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-08 02:55:33,798 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,798 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-08 02:55:33,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:33,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-08 02:55:33,810 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-08 02:55:33,810 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:33,810 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:33,810 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-08 02:55:33,810 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:33,818 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,818 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,818 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,818 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,818 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:33,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:33,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,819 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,819 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:33,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,821 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,822 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:33,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,826 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:33,827 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-08 02:55:33,827 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-08 02:55:33,840 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,840 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,839 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,840 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:33,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:33,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:33,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:33,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,841 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:33,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-08 02:55:33,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:33,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:33,841 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:33,842 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,842 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:33,842 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,842 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-08 02:55:33,842 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-08 02:55:33,941 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:33,942 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:33,942 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:33,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:33,961 INFO [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:33,962 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 because midkey is the same as first or last row 2023-02-08 02:55:33,962 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292 because midkey is the same as first or last row 2023-02-08 02:55:33,963 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegion(2744): Flushing 071ac4e2b6107e64d654a922cf0b1516 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:55:33,994 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/6171808eee3e4f4c8f8c305494a78fe5 2023-02-08 02:55:34,003 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/cd97f9f5e8a4462eac0e41644de2d247 2023-02-08 02:55:34,007 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf1/6171808eee3e4f4c8f8c305494a78fe5 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/6171808eee3e4f4c8f8c305494a78fe5 2023-02-08 02:55:34,010 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/6171808eee3e4f4c8f8c305494a78fe5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:34,011 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/.tmp/cf2/cd97f9f5e8a4462eac0e41644de2d247 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/cd97f9f5e8a4462eac0e41644de2d247 2023-02-08 02:55:34,016 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/cd97f9f5e8a4462eac0e41644de2d247, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:34,016 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 071ac4e2b6107e64d654a922cf0b1516 in 53ms, sequenceid=82, compaction requested=true 2023-02-08 02:55:34,017 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.HRegion(2445): Flush status journal for 071ac4e2b6107e64d654a922cf0b1516: 2023-02-08 02:55:34,017 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 because midkey is the same as first or last row 2023-02-08 02:55:34,018 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37605] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292 because midkey is the same as first or last row 2023-02-08 02:55:34,018 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:34,018 DEBUG [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-08 02:55:34,018 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:34,018 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,018 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,020 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,020 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,020 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-08 02:55:34,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testInvalidSplitKey 2023-02-08 02:55:34,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,023 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,023 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 02:55:34,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824934025"}]},"ts":"1675824934025"} 2023-02-08 02:55:34,026 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-08 02:55:34,039 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-08 02:55:34,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, UNASSIGN}] 2023-02-08 02:55:34,040 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, UNASSIGN 2023-02-08 02:55:34,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=071ac4e2b6107e64d654a922cf0b1516, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:34,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675824934041"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824934041"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824934041"}]},"ts":"1675824934041"} 2023-02-08 02:55:34,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 071ac4e2b6107e64d654a922cf0b1516, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:34,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,125 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 02:55:34,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:34,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 071ac4e2b6107e64d654a922cf0b1516, disabling compactions & flushes 2023-02-08 02:55:34,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:34,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:34,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. after waiting 0 ms 2023-02-08 02:55:34,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:34,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:55:34,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:34,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516. 2023-02-08 02:55:34,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 071ac4e2b6107e64d654a922cf0b1516: Waiting for close lock at 1675824934194Running coprocessor pre-close hooks at 1675824934194Disabling compacts and flushes for region at 1675824934194Disabling writes for close at 1675824934195 (+1 ms)Writing region close event to WAL at 1675824934196 (+1 ms)Running coprocessor post-close hooks at 1675824934204 (+8 ms)Closed at 1675824934204 2023-02-08 02:55:34,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:34,207 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=071ac4e2b6107e64d654a922cf0b1516, regionState=CLOSED 2023-02-08 02:55:34,207 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675824934207"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824934207"}]},"ts":"1675824934207"} 2023-02-08 02:55:34,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-08 02:55:34,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 071ac4e2b6107e64d654a922cf0b1516, server=jenkins-hbase12.apache.org,37605,1675824833879 in 166 msec 2023-02-08 02:55:34,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-08 02:55:34,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=071ac4e2b6107e64d654a922cf0b1516, UNASSIGN in 170 msec 2023-02-08 02:55:34,212 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824934212"}]},"ts":"1675824934212"} 2023-02-08 02:55:34,213 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-08 02:55:34,218 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-08 02:55:34,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 198 msec 2023-02-08 02:55:34,328 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,328 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 02:55:34,331 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-08 02:55:34,331 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,332 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testInvalidSplitKey 2023-02-08 02:55:34,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,340 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,340 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,340 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,341 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 02:55:34,344 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:34,347 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/recovered.edits] 2023-02-08 02:55:34,351 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/11a9c11287904e7888a9cb6c1e56ff9e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/11a9c11287904e7888a9cb6c1e56ff9e 2023-02-08 02:55:34,353 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/1c9b05dd26644c1dbe474d692a85ad8e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/1c9b05dd26644c1dbe474d692a85ad8e 2023-02-08 02:55:34,355 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/5ce7d326b5e545da8322b4f913cb26b4 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/5ce7d326b5e545da8322b4f913cb26b4 2023-02-08 02:55:34,356 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/6171808eee3e4f4c8f8c305494a78fe5 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/6171808eee3e4f4c8f8c305494a78fe5 2023-02-08 02:55:34,359 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/8b029a6c1fc94f7793e6e6bf62750a4f to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/8b029a6c1fc94f7793e6e6bf62750a4f 2023-02-08 02:55:34,360 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/bbf222f8544343dda7f8fb6a7c7f0ea6 2023-02-08 02:55:34,362 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/ee5411c74614455ba50f763e5572f8ba to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf1/ee5411c74614455ba50f763e5572f8ba 2023-02-08 02:55:34,368 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/08fa596c3c814098ae1e49753cb0eb25 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/08fa596c3c814098ae1e49753cb0eb25 2023-02-08 02:55:34,370 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/2cab42fc4f8c4687bfc5799df924efee to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/2cab42fc4f8c4687bfc5799df924efee 2023-02-08 02:55:34,373 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/64efed41889348adbade0658eb1df885 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/64efed41889348adbade0658eb1df885 2023-02-08 02:55:34,374 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/88e4b8e85f784baba6d4800872fd91ef to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/88e4b8e85f784baba6d4800872fd91ef 2023-02-08 02:55:34,374 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/cd97f9f5e8a4462eac0e41644de2d247 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/cd97f9f5e8a4462eac0e41644de2d247 2023-02-08 02:55:34,375 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/da542d0d5a454c9395aab6d2526a0292 2023-02-08 02:55:34,376 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/f2ddabb057304d55a04e539d95e3999e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/cf2/f2ddabb057304d55a04e539d95e3999e 2023-02-08 02:55:34,378 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516/recovered.edits/85.seqid 2023-02-08 02:55:34,378 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testInvalidSplitKey/071ac4e2b6107e64d654a922cf0b1516 2023-02-08 02:55:34,378 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-08 02:55:34,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,380 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-08 02:55:34,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-08 02:55:34,385 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,385 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-08 02:55:34,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824934385"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:34,386 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 02:55:34,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 071ac4e2b6107e64d654a922cf0b1516, NAME => 'testInvalidSplitKey,,1675824931756.071ac4e2b6107e64d654a922cf0b1516.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 02:55:34,386 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-08 02:55:34,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824934386"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:34,387 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-08 02:55:34,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:34,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 64 msec 2023-02-08 02:55:34,441 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,442 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-08 02:55:34,443 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-08 02:55:34,466 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=366 (was 347) - Thread LEAK? -, OpenFileDescriptor=742 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=213 (was 206) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=3604 (was 3624) 2023-02-08 02:55:34,481 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=366, OpenFileDescriptor=742, MaxFileDescriptor=60000, SystemLoadAverage=213, ProcessCount=171, AvailableMemoryMB=3602 2023-02-08 02:55:34,482 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:34,482 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:34,483 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:34,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:55:34,492 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:55:34,493 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:34,495 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:55:34,496 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:55:34,498 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,498 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c empty. 2023-02-08 02:55:34,499 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,499 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 02:55:34,513 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-08 02:55:34,514 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 760d9a76a1f3e3afbc3d42d93be1360c, NAME => 'testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 760d9a76a1f3e3afbc3d42d93be1360c, disabling compactions & flushes 2023-02-08 02:55:34,522 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. after waiting 0 ms 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,522 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,522 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 760d9a76a1f3e3afbc3d42d93be1360c: Waiting for close lock at 1675824934522Disabling compacts and flushes for region at 1675824934522Disabling writes for close at 1675824934522Writing region close event to WAL at 1675824934522Closed at 1675824934522 2023-02-08 02:55:34,524 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:55:34,525 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824934525"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824934525"}]},"ts":"1675824934525"} 2023-02-08 02:55:34,526 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:55:34,527 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:55:34,527 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824934527"}]},"ts":"1675824934527"} 2023-02-08 02:55:34,528 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-08 02:55:34,544 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:55:34,546 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:55:34,546 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:55:34,546 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:55:34,546 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:55:34,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, ASSIGN}] 2023-02-08 02:55:34,548 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, ASSIGN 2023-02-08 02:55:34,548 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:34,699 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:55:34,700 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=760d9a76a1f3e3afbc3d42d93be1360c, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:34,700 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824934699"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824934699"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824934699"}]},"ts":"1675824934699"} 2023-02-08 02:55:34,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:34,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 760d9a76a1f3e3afbc3d42d93be1360c, NAME => 'testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:55:34,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:34,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:34,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,858 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,859 DEBUG [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1 2023-02-08 02:55:34,859 DEBUG [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1 2023-02-08 02:55:34,859 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 760d9a76a1f3e3afbc3d42d93be1360c columnFamilyName cf1 2023-02-08 02:55:34,859 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] regionserver.HStore(310): Store=760d9a76a1f3e3afbc3d42d93be1360c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:34,860 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,861 DEBUG [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2 2023-02-08 02:55:34,861 DEBUG [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2 2023-02-08 02:55:34,861 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 760d9a76a1f3e3afbc3d42d93be1360c columnFamilyName cf2 2023-02-08 02:55:34,861 INFO [StoreOpener-760d9a76a1f3e3afbc3d42d93be1360c-1] regionserver.HStore(310): Store=760d9a76a1f3e3afbc3d42d93be1360c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:34,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,864 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:34,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:34,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:55:34,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 760d9a76a1f3e3afbc3d42d93be1360c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74255668, jitterRate=0.1064956784248352}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:34,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 760d9a76a1f3e3afbc3d42d93be1360c: Running coprocessor pre-open hook at 1675824934857Writing region info on filesystem at 1675824934857Initializing all the Stores at 1675824934857Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824934857Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824934857Cleaning up temporary data from old regions at 1675824934863 (+6 ms)Cleaning up detritus from prior splits at 1675824934863Running coprocessor post-open hooks at 1675824934867 (+4 ms)Region opened successfully at 1675824934867 2023-02-08 02:55:34,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c., pid=111, masterSystemTime=1675824934853 2023-02-08 02:55:34,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:34,869 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=760d9a76a1f3e3afbc3d42d93be1360c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:34,869 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824934869"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824934869"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824934869"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824934869"}]},"ts":"1675824934869"} 2023-02-08 02:55:34,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-08 02:55:34,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 in 169 msec 2023-02-08 02:55:34,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-08 02:55:34,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, ASSIGN in 325 msec 2023-02-08 02:55:34,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:55:34,873 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824934873"}]},"ts":"1675824934873"} 2023-02-08 02:55:34,874 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-08 02:55:34,924 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:55:34,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 431 msec 2023-02-08 02:55:35,000 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,000 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:35,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:35,002 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,002 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,002 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:35,002 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:35,003 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,003 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,049 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,050 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,050 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,051 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,051 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,052 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,052 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,052 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,052 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:35,052 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:35,052 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:35,054 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:55:35,060 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,062 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 2023-02-08 02:55:35,070 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,070 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,071 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,071 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,071 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,077 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/c610ca09c4ac47f698875c5e4c839a65 2023-02-08 02:55:35,081 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 2023-02-08 02:55:35,084 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:35,085 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/c610ca09c4ac47f698875c5e4c839a65 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 2023-02-08 02:55:35,089 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:35,090 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 37ms, sequenceid=6, compaction requested=false 2023-02-08 02:55:35,090 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,090 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:35,090 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,090 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:35,090 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:35,090 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,091 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,091 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,102 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,102 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,103 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,103 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:35,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d74e2f4[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:35,105 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:35,105 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,113 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:35,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,113 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:35,123 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,123 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,123 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,134 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,134 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,134 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,144 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,144 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,144 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,144 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:35,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:35,148 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:35,148 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:35,148 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:35,148 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,148 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,155 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,156 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,157 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,157 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,157 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:35,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,165 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,166 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,166 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,267 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:35,267 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,267 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:35,287 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,287 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:35,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:35,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:35,290 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,290 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,290 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:35,290 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:35,291 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,291 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,302 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,302 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,302 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,302 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,304 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,304 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:35,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,306 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,306 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,306 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,306 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,307 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,307 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,307 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,309 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,309 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,309 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,309 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:35,310 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,310 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,310 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,310 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:35,319 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/9e501b9d143147d195d533a89be5d9d8 2023-02-08 02:55:35,329 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/d019281617c14eadbe4422c06c4efa45 2023-02-08 02:55:35,333 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/9e501b9d143147d195d533a89be5d9d8 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 2023-02-08 02:55:35,337 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:35,338 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/d019281617c14eadbe4422c06c4efa45 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 2023-02-08 02:55:35,342 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:35,342 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 32ms, sequenceid=19, compaction requested=false 2023-02-08 02:55:35,343 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:35,343 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,343 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,364 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,364 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,364 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,376 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,376 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,376 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,376 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:35,378 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@349af3b[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:35,378 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:35,378 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:35,397 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:35,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,397 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,398 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,398 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,398 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,398 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:35,398 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:35,501 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,501 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,501 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,575 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:35,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:35,582 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:35,582 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:35,583 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:35,583 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,583 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,597 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,597 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,598 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,598 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,599 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,599 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,599 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:35,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,618 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,719 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:35,719 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,719 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:35,732 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,732 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:35,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:35,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:35,734 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,734 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,734 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:35,734 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:35,734 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,734 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,744 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,745 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,745 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,747 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:35,746 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,747 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,747 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,747 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,747 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:35,748 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:35,755 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,755 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,755 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,755 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,755 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,756 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,756 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/140afc9e16894cb5984ed7fa0132e3ab 2023-02-08 02:55:35,768 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/9ff6cc273faf43d093bfc9e3818f7560 2023-02-08 02:55:35,772 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/140afc9e16894cb5984ed7fa0132e3ab as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab 2023-02-08 02:55:35,775 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:35,775 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/9ff6cc273faf43d093bfc9e3818f7560 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 2023-02-08 02:55:35,778 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:35,779 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 32ms, sequenceid=32, compaction requested=true 2023-02-08 02:55:35,779 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:35,779 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:35,779 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:35,779 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:35,779 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,779 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:35,779 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:35,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,786 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,787 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,787 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:35,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:35,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f7429a[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:35,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:35,789 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:35,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:35,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,798 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,797 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:35,798 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:35,798 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:35,798 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:35,807 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,807 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,807 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,850 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,850 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:35,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:35,856 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:35,856 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:35,856 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:35,856 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:35,856 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:35,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,872 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:35,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,873 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,874 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:35,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,878 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:35,878 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:35,879 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:35,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:35,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,892 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:35,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:35,892 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,893 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:35,893 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:35,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:35,994 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:35,994 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:35,994 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:35,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:36,011 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,011 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:36,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:36,014 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:36,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:36,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,014 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,027 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,028 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,028 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:36,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,031 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,032 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,033 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:36,033 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:36,033 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,033 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:36,033 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:36,039 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,045 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/ad5e815b62fa4eab943ef5992fc5fb14 2023-02-08 02:55:36,049 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,050 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,059 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/8f34f56669ae4545a43229127e97d991 2023-02-08 02:55:36,064 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/ad5e815b62fa4eab943ef5992fc5fb14 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 2023-02-08 02:55:36,070 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:36,071 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/8f34f56669ae4545a43229127e97d991 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 2023-02-08 02:55:36,076 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:36,077 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 43ms, sequenceid=45, compaction requested=true 2023-02-08 02:55:36,077 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:36,077 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,077 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:36,077 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:36,077 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,077 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,077 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,090 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,090 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,091 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,091 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:36,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70e3683e[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:36,097 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:36,097 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,102 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,102 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,102 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,103 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,104 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,104 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:36,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,103 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,104 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:36,103 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,104 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:36,104 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,104 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,105 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:36,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,113 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,124 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,124 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,124 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,134 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,134 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,134 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,134 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:36,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:36,147 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:36,147 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:36,147 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:36,147 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,147 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,155 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,156 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,158 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,158 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,162 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,164 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,176 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,176 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,277 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:36,277 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,277 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:36,299 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,299 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:36,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:36,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:36,302 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,302 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,302 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:36,302 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:36,303 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,303 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,323 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,323 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,323 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,323 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,324 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,324 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:36,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,326 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,326 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,327 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,327 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:36,327 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,328 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,327 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,328 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,328 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,328 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,328 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:36,328 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,328 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:36,341 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/028a58f236824295b9a245455138a7a7 2023-02-08 02:55:36,358 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/6669fa63f61f4763a1b6d3e9a1447882 2023-02-08 02:55:36,362 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/028a58f236824295b9a245455138a7a7 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 2023-02-08 02:55:36,366 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:36,367 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/6669fa63f61f4763a1b6d3e9a1447882 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 2023-02-08 02:55:36,372 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:36,373 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 45ms, sequenceid=58, compaction requested=true 2023-02-08 02:55:36,373 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:36,374 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,374 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:36,374 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:36,374 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,374 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,374 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,390 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,418 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,418 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,428 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,428 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,428 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,429 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,429 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:36,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a8a5794[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:36,431 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:36,431 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,439 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,439 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,439 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,439 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:36,439 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:36,439 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,440 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:36,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,440 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,440 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,440 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:36,449 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,449 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,450 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,449 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,450 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,450 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,460 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,460 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,460 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:36,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:36,466 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:36,466 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:36,466 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:36,467 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,467 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:36,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,481 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,481 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,481 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,482 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,483 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,487 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,487 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,488 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:36,503 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,503 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,503 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,504 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,526 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 506934cb-153f-48e7-8277-804b83b91344) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:36,604 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:36,605 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,605 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:36,615 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,615 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:36,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:36,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-08 02:55:36,617 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-08 02:55:36,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-08 02:55:36,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,617 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,628 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,628 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,629 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,629 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:36,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:36,630 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,631 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,631 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. started... 2023-02-08 02:55:36,631 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,631 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:36,639 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,639 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/313c6d3b051e4ccd953c2cb947a3ec38 2023-02-08 02:55:36,649 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,649 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,650 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,650 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,650 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,653 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/792a8b2297e44bcfbf35e53022f531fb 2023-02-08 02:55:36,659 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/313c6d3b051e4ccd953c2cb947a3ec38 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 2023-02-08 02:55:36,664 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:36,665 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/792a8b2297e44bcfbf35e53022f531fb as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb 2023-02-08 02:55:36,670 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:36,671 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 40ms, sequenceid=71, compaction requested=true 2023-02-08 02:55:36,671 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 760d9a76a1f3e3afbc3d42d93be1360c: 2023-02-08 02:55:36,671 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:36,671 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:36,671 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:36,671 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,671 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-08 02:55:36,671 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-08 02:55:36,681 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,681 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,682 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,682 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:36,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-08 02:55:36,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34740f89[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:36,687 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-08 02:55:36,687 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,692 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,693 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:36,692 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:36,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:36,693 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:36,702 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,702 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,703 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,739 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,739 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,740 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,740 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,739 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-08 02:55:36,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,740 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-08 02:55:36,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:36,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-08 02:55:36,753 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-08 02:55:36,753 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:36,754 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:36,754 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-08 02:55:36,754 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:36,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:36,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,777 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:36,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,779 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,780 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:36,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,783 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:36,784 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-08 02:55:36,784 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,797 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-08 02:55:36,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,798 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:36,797 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:36,798 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:36,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,798 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,798 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,799 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:36,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:36,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,799 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,799 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:36,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,800 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-08 02:55:36,899 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:36,899 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:36,899 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:36,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:37,100 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:55:37,158 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-08 02:55:37,159 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:37,159 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:37,161 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:37,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:37,410 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:37,410 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:37,411 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:37,411 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:37,411 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:37,413 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-08 02:55:37,413 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:37,414 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:37,414 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:37,414 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:37,414 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,415 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:37,415 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,415 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,415 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,416 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:37,416 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,416 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,416 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,416 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:37,417 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:37,417 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,417 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,417 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,417 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,418 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,419 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,419 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,419 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:37,420 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,420 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,420 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,420 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,420 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,421 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:37,421 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,421 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:37,421 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,421 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,421 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,421 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 02:55:37,422 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:37,423 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:37,423 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824967423 2023-02-08 02:55:37,425 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,425 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:37,427 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,428 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,428 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:37,428 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824997428 2023-02-08 02:55:37,428 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825057428 2023-02-08 02:55:37,428 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 02:55:37,428 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:37,430 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:37,430 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:37,430 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:37,678 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,678 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:37,678 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:37,678 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:37,679 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:37,679 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:37,679 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:37,679 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:37,679 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:37,680 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:37,682 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-08 02:55:37,682 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:37,682 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:37,683 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:37,683 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:37,683 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,683 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:37,683 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,684 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,684 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,684 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:37,684 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,684 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,685 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,685 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:37,685 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:37,685 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,686 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,686 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,686 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:37,686 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,686 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,687 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,687 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:37,687 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,687 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,688 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,688 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,688 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:37,688 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,688 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,689 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,689 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,689 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,689 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:37,690 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,690 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:37,690 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,690 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,690 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,690 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-08 02:55:37,691 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:37,691 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:37,692 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824967692 2023-02-08 02:55:37,692 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:37,693 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,695 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,695 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,695 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:37,695 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824997695 2023-02-08 02:55:37,695 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825057695 2023-02-08 02:55:37,695 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:55:37,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:37,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN}] 2023-02-08 02:55:37,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:37,698 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:37,698 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:37,946 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,946 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:37,946 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:37,946 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:37,946 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:37,946 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:37,947 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:37,947 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:37,947 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:37,948 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:37,949 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-08 02:55:37,950 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:37,950 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:37,950 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:37,951 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:37,951 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,951 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:37,951 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,952 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:37,952 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,952 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:37,952 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,953 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:37,953 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,953 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:37,954 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:37,954 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,954 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:37,954 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,955 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:37,955 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,955 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:37,956 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,956 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:37,957 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,957 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:37,957 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,957 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,958 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:37,958 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,958 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:37,958 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,959 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,959 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:37,959 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:37,959 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,960 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:37,960 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,960 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,960 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,960 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-08 02:55:37,960 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:37,961 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:37,963 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824967963 2023-02-08 02:55:37,963 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:37,964 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:37,966 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,966 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:37,966 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:37,966 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824997966 2023-02-08 02:55:37,966 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825057966 2023-02-08 02:55:37,967 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:55:37,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN}] 2023-02-08 02:55:37,967 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:37,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:37,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:37,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824937969"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824937969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824937969"}]},"ts":"1675824937969"} 2023-02-08 02:55:37,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:38,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,123 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 760d9a76a1f3e3afbc3d42d93be1360c, disabling compactions & flushes 2023-02-08 02:55:38,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:38,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:38,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. after waiting 0 ms 2023-02-08 02:55:38,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:38,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 760d9a76a1f3e3afbc3d42d93be1360c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:55:38,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/c9e62bb01b4c411e82efb7fe1a258a73 2023-02-08 02:55:38,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b 2023-02-08 02:55:38,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf1/c9e62bb01b4c411e82efb7fe1a258a73 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 2023-02-08 02:55:38,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:38,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/.tmp/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b 2023-02-08 02:55:38,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:38,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 42ms, sequenceid=82, compaction requested=true 2023-02-08 02:55:38,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:55:38,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:38,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. 2023-02-08 02:55:38,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 760d9a76a1f3e3afbc3d42d93be1360c: Waiting for close lock at 1675824938123Running coprocessor pre-close hooks at 1675824938123Disabling compacts and flushes for region at 1675824938123Disabling writes for close at 1675824938124 (+1 ms)Obtaining lock to block concurrent updates at 1675824938124Preparing flush snapshotting stores in 760d9a76a1f3e3afbc3d42d93be1360c at 1675824938124Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824938125 (+1 ms)Flushing stores of testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. at 1675824938126 (+1 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf1: creating writer at 1675824938127 (+1 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf1: appending metadata at 1675824938129 (+2 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf1: closing flushed file at 1675824938129Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf2: creating writer at 1675824938146 (+17 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf2: appending metadata at 1675824938147 (+1 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf2: closing flushed file at 1675824938147Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf1: reopening flushed file at 1675824938159 (+12 ms)Flushing 760d9a76a1f3e3afbc3d42d93be1360c/cf2: reopening flushed file at 1675824938163 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 760d9a76a1f3e3afbc3d42d93be1360c in 42ms, sequenceid=82, compaction requested=true at 1675824938166 (+3 ms)Writing region close event to WAL at 1675824938167 (+1 ms)Running coprocessor post-close hooks at 1675824938170 (+3 ms)Closed at 1675824938171 (+1 ms) 2023-02-08 02:55:38,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,174 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED 2023-02-08 02:55:38,174 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824938173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824938173"}]},"ts":"1675824938173"} 2023-02-08 02:55:38,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-08 02:55:38,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 in 204 msec 2023-02-08 02:55:38,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-08 02:55:38,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN in 209 msec 2023-02-08 02:55:38,180 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:38,180 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:38,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:38,217 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,217 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:38,217 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:38,217 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:38,218 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:38,218 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:38,218 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:38,218 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:38,218 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:38,219 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:55:38,222 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-08 02:55:38,222 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:38,222 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:38,223 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:38,224 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:38,224 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:38,224 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:38,225 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,225 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:38,225 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,226 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,226 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,226 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:38,227 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,227 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,227 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,228 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:38,228 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:38,228 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,229 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:38,230 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:38,231 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:38,232 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,232 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,232 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,232 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:38,232 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:38,232 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-08 02:55:38,232 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:38,233 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:38,233 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824968233 2023-02-08 02:55:38,234 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:38,234 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,236 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:38,236 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:38,236 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:38,236 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824998236 2023-02-08 02:55:38,237 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825058237 2023-02-08 02:55:38,237 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:55:38,239 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:38,248 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=760d9a76a1f3e3afbc3d42d93be1360c, threads=14 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,248 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,249 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,348 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,359 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,361 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,366 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,369 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,382 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,383 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,383 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,384 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,384 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,384 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,385 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,385 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,387 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,387 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 760d9a76a1f3e3afbc3d42d93be1360c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:55:38,417 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:38,417 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:38,417 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:38,487 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,487 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:38,487 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:38,488 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:38,488 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:38,488 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:38,488 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:38,489 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:38,489 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:38,490 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:55:38,493 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-08 02:55:38,494 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:38,494 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:38,495 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:38,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:38,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:38,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:38,497 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,498 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:38,498 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,499 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:38,499 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,500 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:38,500 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,500 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:38,500 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:38,501 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:38,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:38,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:38,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:38,504 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,504 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,504 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:38,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:38,504 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-02-08 02:55:38,505 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:38,505 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:38,505 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824968505 2023-02-08 02:55:38,507 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:38,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:38,509 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:38,509 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:38,509 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:38,509 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824998509 2023-02-08 02:55:38,509 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825058509 2023-02-08 02:55:38,509 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:55:38,518 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=760d9a76a1f3e3afbc3d42d93be1360c, threads=14 2023-02-08 02:55:38,518 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,518 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,518 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,519 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,520 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,604 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,615 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,622 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,623 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,623 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,624 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,624 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,629 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,629 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,633 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,633 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,634 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:38,988 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:39,025 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 for region: 760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:39,026 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 760d9a76a1f3e3afbc3d42d93be1360c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:55:39,076 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:39,080 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:39,081 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:39,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:39,081 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:39,081 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:39,260 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,260 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:39,260 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:39,260 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:39,261 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:39,261 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:39,261 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:39,262 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:39,262 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:39,263 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:39,265 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-08 02:55:39,265 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:39,265 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:39,265 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:39,266 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:39,266 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:39,266 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:39,266 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,266 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:39,267 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,267 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,267 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,267 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:39,268 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,268 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,268 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,268 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:39,268 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:39,269 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,269 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,269 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,269 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:39,269 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,270 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,270 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,270 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:39,270 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,270 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,271 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,272 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,272 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,272 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:39,272 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,272 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:39,273 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:39,273 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:39,273 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,273 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,273 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,273 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,273 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,273 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-08 02:55:39,274 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:39,274 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:39,279 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824969279 2023-02-08 02:55:39,280 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:39,280 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,282 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,282 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,282 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:39,282 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824999282 2023-02-08 02:55:39,282 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825059282 2023-02-08 02:55:39,282 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:55:39,285 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:39,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:39,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:39,287 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:39,533 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,533 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:39,533 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:39,533 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:39,533 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:39,534 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:39,534 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:39,534 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:39,534 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:39,535 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:55:39,537 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-08 02:55:39,537 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:39,538 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:39,538 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:39,538 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:39,538 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:39,539 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:39,539 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,539 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:39,539 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,540 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,540 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,540 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:39,540 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,541 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,541 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,541 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:39,542 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:39,542 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,542 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,542 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,542 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:39,543 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,543 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,543 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,543 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:39,544 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,544 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,544 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,544 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,545 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:39,545 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,545 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,546 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,546 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,546 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,546 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:39,547 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,547 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,547 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,547 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,547 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-08 02:55:39,548 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:39,548 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:39,549 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824969549 2023-02-08 02:55:39,550 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:39,550 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,552 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,552 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,553 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:39,553 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824999553 2023-02-08 02:55:39,553 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825059553 2023-02-08 02:55:39,553 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-08 02:55:39,553 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:39,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824939555"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675824939555"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675824939555"}]},"ts":"1675824939555"} 2023-02-08 02:55:39,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824939555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824939555"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824939555"}]},"ts":"1675824939555"} 2023-02-08 02:55:39,555 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824939555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824939555"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824939555"}]},"ts":"1675824939555"} 2023-02-08 02:55:39,557 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:39,557 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:39,557 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:39,803 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,803 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:39,803 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:39,804 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:39,804 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:39,804 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:39,805 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:39,805 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:39,805 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:39,807 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:55:39,810 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-08 02:55:39,810 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:39,810 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:39,811 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:39,812 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:39,812 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:39,812 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:39,813 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,814 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:39,814 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,814 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:39,815 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,815 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:39,815 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,815 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:39,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:39,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:39,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:39,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:39,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:39,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:39,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:39,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:39,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:39,822 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,822 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,822 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,822 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,822 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:39,822 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 15 msec 2023-02-08 02:55:39,823 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:39,823 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:39,824 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824969824 2023-02-08 02:55:39,824 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:39,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,827 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,827 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:39,828 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:39,828 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=45371de536b6594f3d1730c5e4920f27, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:39,828 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=becfd878a7a2594e71d99031c61388c1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:39,828 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:39,828 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675824999828 2023-02-08 02:55:39,828 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825059828 2023-02-08 02:55:39,828 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-08 02:55:39,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675824939829"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675824939829"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675824939829"}]},"ts":"1675824939829"} 2023-02-08 02:55:39,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824939829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824939829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824939829"}]},"ts":"1675824939829"} 2023-02-08 02:55:39,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824939829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824939829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824939829"}]},"ts":"1675824939829"} 2023-02-08 02:55:39,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:39,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:39,833 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:39,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:39,833 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:40,079 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:40,079 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:40,080 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:40,080 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:40,080 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:40,081 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:40,081 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:40,081 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:40,082 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 02:55:40,084 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-08 02:55:40,084 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:40,085 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:40,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:40,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:40,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:40,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:40,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:40,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,088 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:40,088 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,088 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,088 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,089 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,090 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,091 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:40,091 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,091 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,091 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,092 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,092 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,092 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 02:55:40,092 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:40,092 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:40,093 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824970093 2023-02-08 02:55:40,093 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:40,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,097 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,097 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,097 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:40,098 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=45371de536b6594f3d1730c5e4920f27, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:40,098 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=becfd878a7a2594e71d99031c61388c1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:40,098 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:40,098 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675825000098 2023-02-08 02:55:40,098 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825060098 2023-02-08 02:55:40,098 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-08 02:55:40,098 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:40,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN}] 2023-02-08 02:55:40,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:40,101 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:40,101 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:40,348 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,349 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:40,349 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:40,349 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:40,349 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:40,349 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:40,349 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:40,350 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:40,350 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:40,351 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:40,353 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-08 02:55:40,353 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:40,354 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:40,354 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:40,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:40,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:40,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:40,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,356 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:40,356 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,356 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,356 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:40,360 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,360 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,360 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,361 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 02:55:40,361 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:40,362 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:40,362 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824970362 2023-02-08 02:55:40,363 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:40,364 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,366 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,366 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,366 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:40,366 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=45371de536b6594f3d1730c5e4920f27, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:40,366 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=becfd878a7a2594e71d99031c61388c1, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-08 02:55:40,367 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:40,367 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675825000367 2023-02-08 02:55:40,367 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825060367 2023-02-08 02:55:40,367 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:55:40,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN}] 2023-02-08 02:55:40,367 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:40,369 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN 2023-02-08 02:55:40,370 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:40,371 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN 2023-02-08 02:55:40,371 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN; state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:40,520 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:55:40,522 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=45371de536b6594f3d1730c5e4920f27, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,522 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824940522"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824940522"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824940522"}]},"ts":"1675824940522"} 2023-02-08 02:55:40,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:40,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=becfd878a7a2594e71d99031c61388c1, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,527 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824940526"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824940526"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824940526"}]},"ts":"1675824940526"} 2023-02-08 02:55:40,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:40,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 45371de536b6594f3d1730c5e4920f27, NAME => 'testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 02:55:40,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:40,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,687 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,689 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1 2023-02-08 02:55:40,689 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1 2023-02-08 02:55:40,689 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45371de536b6594f3d1730c5e4920f27 columnFamilyName cf1 2023-02-08 02:55:40,707 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-bottom 2023-02-08 02:55:40,713 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-bottom 2023-02-08 02:55:40,720 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-bottom 2023-02-08 02:55:40,724 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-bottom 2023-02-08 02:55:40,727 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-bottom 2023-02-08 02:55:40,731 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-bottom 2023-02-08 02:55:40,734 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-bottom 2023-02-08 02:55:40,734 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(310): Store=45371de536b6594f3d1730c5e4920f27/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:40,734 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,735 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2 2023-02-08 02:55:40,735 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2 2023-02-08 02:55:40,735 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 45371de536b6594f3d1730c5e4920f27 columnFamilyName cf2 2023-02-08 02:55:40,746 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-bottom 2023-02-08 02:55:40,751 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-bottom 2023-02-08 02:55:40,755 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-bottom 2023-02-08 02:55:40,759 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-bottom 2023-02-08 02:55:40,763 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-bottom 2023-02-08 02:55:40,767 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-bottom 2023-02-08 02:55:40,771 DEBUG [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-bottom 2023-02-08 02:55:40,771 INFO [StoreOpener-45371de536b6594f3d1730c5e4920f27-1] regionserver.HStore(310): Store=45371de536b6594f3d1730c5e4920f27/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:40,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:40,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:40,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 45371de536b6594f3d1730c5e4920f27; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59383970, jitterRate=-0.1151098906993866}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:40,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 45371de536b6594f3d1730c5e4920f27: Running coprocessor pre-open hook at 1675824940685Writing region info on filesystem at 1675824940685Initializing all the Stores at 1675824940687 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824940687Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824940687Cleaning up temporary data from old regions at 1675824940773 (+86 ms)Cleaning up detritus from prior splits at 1675824940773Running coprocessor post-open hooks at 1675824940775 (+2 ms)Region opened successfully at 1675824940775 2023-02-08 02:55:40,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27., pid=117, masterSystemTime=1675824940676 2023-02-08 02:55:40,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:40,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:40,777 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:40,777 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:40,777 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,777 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): 45371de536b6594f3d1730c5e4920f27/cf1 is initiating minor compaction (all files) 2023-02-08 02:55:40,777 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,777 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 45371de536b6594f3d1730c5e4920f27/cf1 in testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,777 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): 45371de536b6594f3d1730c5e4920f27/cf2 is initiating minor compaction (all files) 2023-02-08 02:55:40,777 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 45371de536b6594f3d1730c5e4920f27/cf2 in testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,778 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/.tmp, totalSize=34.9 K 2023-02-08 02:55:40,778 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/.tmp, totalSize=34.9 K 2023-02-08 02:55:40,778 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824934996 2023-02-08 02:55:40,778 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824934996 2023-02-08 02:55:40,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:40,779 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824935270 2023-02-08 02:55:40,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,779 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824935270 2023-02-08 02:55:40,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => becfd878a7a2594e71d99031c61388c1, NAME => 'testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 02:55:40,779 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=45371de536b6594f3d1730c5e4920f27, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824940779"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824940779"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824940779"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824940779"}]},"ts":"1675824940779"} 2023-02-08 02:55:40,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:40,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:40,780 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824935721 2023-02-08 02:55:40,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,780 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824935721 2023-02-08 02:55:40,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,780 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824935996 2023-02-08 02:55:40,780 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824935996 2023-02-08 02:55:40,781 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,781 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824936281 2023-02-08 02:55:40,781 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824936281 2023-02-08 02:55:40,782 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824936606 2023-02-08 02:55:40,782 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824936606 2023-02-08 02:55:40,782 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1 2023-02-08 02:55:40,782 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1 2023-02-08 02:55:40,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-08 02:55:40,782 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824936901 2023-02-08 02:55:40,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879 in 256 msec 2023-02-08 02:55:40,782 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region becfd878a7a2594e71d99031c61388c1 columnFamilyName cf1 2023-02-08 02:55:40,782 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824936901 2023-02-08 02:55:40,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN in 415 msec 2023-02-08 02:55:40,795 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 45371de536b6594f3d1730c5e4920f27#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:40,796 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 45371de536b6594f3d1730c5e4920f27#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:40,797 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-top 2023-02-08 02:55:40,803 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-top 2023-02-08 02:55:40,810 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-top 2023-02-08 02:55:40,814 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-top 2023-02-08 02:55:40,815 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/.tmp/cf2/36877a5631d14c5a85beab71f21bd3fa as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/36877a5631d14c5a85beab71f21bd3fa 2023-02-08 02:55:40,816 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-top 2023-02-08 02:55:40,817 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/.tmp/cf1/0e23874ad3ca4f10955778e4d186185a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/0e23874ad3ca4f10955778e4d186185a 2023-02-08 02:55:40,819 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-top 2023-02-08 02:55:40,819 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 45371de536b6594f3d1730c5e4920f27/cf2 of 45371de536b6594f3d1730c5e4920f27 into 36877a5631d14c5a85beab71f21bd3fa(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:40,819 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 45371de536b6594f3d1730c5e4920f27: 2023-02-08 02:55:40,820 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27., storeName=45371de536b6594f3d1730c5e4920f27/cf2, priority=9, startTime=1675824940777; duration=0sec 2023-02-08 02:55:40,820 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:40,821 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 45371de536b6594f3d1730c5e4920f27/cf1 of 45371de536b6594f3d1730c5e4920f27 into 0e23874ad3ca4f10955778e4d186185a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:40,821 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 45371de536b6594f3d1730c5e4920f27: 2023-02-08 02:55:40,822 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27., storeName=45371de536b6594f3d1730c5e4920f27/cf1, priority=9, startTime=1675824940776; duration=0sec 2023-02-08 02:55:40,822 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:40,824 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-top 2023-02-08 02:55:40,824 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(310): Store=becfd878a7a2594e71d99031c61388c1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:40,824 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,824 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2 2023-02-08 02:55:40,824 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2 2023-02-08 02:55:40,825 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region becfd878a7a2594e71d99031c61388c1 columnFamilyName cf2 2023-02-08 02:55:40,835 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-top 2023-02-08 02:55:40,838 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-top 2023-02-08 02:55:40,841 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-top 2023-02-08 02:55:40,844 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-top 2023-02-08 02:55:40,847 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-top 2023-02-08 02:55:40,849 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-top 2023-02-08 02:55:40,852 DEBUG [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-top 2023-02-08 02:55:40,852 INFO [StoreOpener-becfd878a7a2594e71d99031c61388c1-1] regionserver.HStore(310): Store=becfd878a7a2594e71d99031c61388c1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:40,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:40,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened becfd878a7a2594e71d99031c61388c1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67602427, jitterRate=0.007354661822319031}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:40,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for becfd878a7a2594e71d99031c61388c1: Running coprocessor pre-open hook at 1675824940780Writing region info on filesystem at 1675824940780Initializing all the Stores at 1675824940781 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824940781Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824940781Cleaning up temporary data from old regions at 1675824940854 (+73 ms)Cleaning up detritus from prior splits at 1675824940854Running coprocessor post-open hooks at 1675824940856 (+2 ms)Region opened successfully at 1675824940856 2023-02-08 02:55:40,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1., pid=118, masterSystemTime=1675824940676 2023-02-08 02:55:40,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:40,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:40,858 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:40,858 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:40,858 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,858 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,858 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): becfd878a7a2594e71d99031c61388c1/cf1 is initiating minor compaction (all files) 2023-02-08 02:55:40,858 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): becfd878a7a2594e71d99031c61388c1/cf2 is initiating minor compaction (all files) 2023-02-08 02:55:40,858 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of becfd878a7a2594e71d99031c61388c1/cf1 in testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,858 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of becfd878a7a2594e71d99031c61388c1/cf2 in testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,859 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/.tmp, totalSize=34.9 K 2023-02-08 02:55:40,859 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/.tmp, totalSize=34.9 K 2023-02-08 02:55:40,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:40,859 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824934996 2023-02-08 02:55:40,859 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824934996 2023-02-08 02:55:40,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=becfd878a7a2594e71d99031c61388c1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,859 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824935270 2023-02-08 02:55:40,859 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824935270 2023-02-08 02:55:40,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824940859"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824940859"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824940859"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824940859"}]},"ts":"1675824940859"} 2023-02-08 02:55:40,860 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824935721 2023-02-08 02:55:40,860 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824935721 2023-02-08 02:55:40,860 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824935996 2023-02-08 02:55:40,860 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824935996 2023-02-08 02:55:40,861 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824936281 2023-02-08 02:55:40,861 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824936281 2023-02-08 02:55:40,861 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824936606 2023-02-08 02:55:40,862 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824936606 2023-02-08 02:55:40,862 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824936901 2023-02-08 02:55:40,862 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824936901 2023-02-08 02:55:40,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-08 02:55:40,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879 in 333 msec 2023-02-08 02:55:40,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-08 02:55:40,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN in 497 msec 2023-02-08 02:55:40,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-08 02:55:40,867 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:40,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-08 02:55:40,867 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,867 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-08 02:55:40,867 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:40,867 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:40,868 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-08 02:55:40,869 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-08 02:55:40,869 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-08 02:55:40,869 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-08 02:55:40,869 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 02:55:40,869 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-08 02:55:40,870 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-08 02:55:40,871 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=61b3f9480f07089e1b2877d5f55bdd46, daughterA=c97fd6e93be08968b8a2ca2719084f7a, daughterB=7e254f706c3041846348609ee0d53549 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,872 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=edae7e6dcabc9beb6fca230c3a7df74c, daughterA=4be5e4800c36e1a14f75c459c9ec0f6a, daughterB=21c6f7f548611c6f9d9be71c188e4c15 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,873 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=a6003dd11a06ff730283e23c33e29876, daughterA=20c6efed62991c27b5a3a43523a99902, daughterB=553b6d198ab5781a750a6fe711553d74 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=a6003dd11a06ff730283e23c33e29876, ASSIGN 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7021d1e7a9deec290548f73330a10890, daughterA=d4cc7e9a06f1fc94b8cfc192b9959729, daughterB=856866739a370df8d72a12d9a5e896ab 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,874 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=833776ad2367c20696c3f51f3ecac4a3, daughterA=b1f619421cec5a6184b63bb1631f4a72, daughterB=2a409a2959c68e7714a42d395c76489c 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,875 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRollbackForSplitTableRegionWithReplica, region=39b49a9b2047bfbdfdf109b4bad2ab6d is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=5dfcb5ec6b56adf0449fc6c004177ac9, daughterA=6b28c82e7292267e4bfb412a6547d372, daughterB=de2a8e0e6fa5fe4e102175fb004c7fb6 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-08 02:55:40,876 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): becfd878a7a2594e71d99031c61388c1#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:40,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:40,876 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): becfd878a7a2594e71d99031c61388c1#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:40,877 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 held the lock before restarting, call acquireLock to restore it. 2023-02-08 02:55:40,878 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,878 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,878 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 760d9a76a1f3e3afbc3d42d93be1360c, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,878 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,879 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879 didn't hold the lock before restarting, skip acquiring lock. 2023-02-08 02:55:40,879 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-08 02:55:40,883 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-08 02:55:40,884 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase12.apache.org,46783,1675824833788, table=hbase:meta, region=1588230740 2023-02-08 02:55:40,892 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675824970892 2023-02-08 02:55:40,893 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:40,893 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 02:55:40,896 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=1dababf94e1d1ac62f98f642609de9f5, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,896 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=760d9a76a1f3e3afbc3d42d93be1360c, regionState=CLOSED, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=2 2023-02-08 02:55:40,896 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase12.apache.org,37605,1675824833879, table=testRecoveryAndDoubleExecution, region=760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:40,896 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=45371de536b6594f3d1730c5e4920f27, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=86 2023-02-08 02:55:40,897 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=becfd878a7a2594e71d99031c61388c1, regionState=OPEN, lastHost=jenkins-hbase12.apache.org,37605,1675824833879, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879, openSeqNum=86 2023-02-08 02:55:40,897 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-08 02:55:40,897 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675825000897 2023-02-08 02:55:40,897 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675825060897 2023-02-08 02:55:40,897 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-08 02:55:40,901 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-08 02:55:40,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=760d9a76a1f3e3afbc3d42d93be1360c, daughterA=45371de536b6594f3d1730c5e4920f27, daughterB=becfd878a7a2594e71d99031c61388c1 in 3.7420 sec 2023-02-08 02:55:40,904 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/.tmp/cf1/e84e80ec31c945978fa8fd4f566b05af as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/e84e80ec31c945978fa8fd4f566b05af 2023-02-08 02:55:40,906 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/.tmp/cf2/c8b130ed70c04a9e9f62d1dfe92cb7dc as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c8b130ed70c04a9e9f62d1dfe92cb7dc 2023-02-08 02:55:40,908 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in becfd878a7a2594e71d99031c61388c1/cf1 of becfd878a7a2594e71d99031c61388c1 into e84e80ec31c945978fa8fd4f566b05af(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:40,909 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for becfd878a7a2594e71d99031c61388c1: 2023-02-08 02:55:40,909 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1., storeName=becfd878a7a2594e71d99031c61388c1/cf1, priority=9, startTime=1675824940857; duration=0sec 2023-02-08 02:55:40,909 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:40,910 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in becfd878a7a2594e71d99031c61388c1/cf2 of becfd878a7a2594e71d99031c61388c1 into c8b130ed70c04a9e9f62d1dfe92cb7dc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:40,910 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for becfd878a7a2594e71d99031c61388c1: 2023-02-08 02:55:40,910 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1., storeName=becfd878a7a2594e71d99031c61388c1/cf2, priority=9, startTime=1675824940858; duration=0sec 2023-02-08 02:55:40,910 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:41,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37605] ipc.CallRunner(144): callId: 860 service: ClientService methodName: Scan size: 154 connection: 136.243.104.168:32972 deadline: 1675825001152, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c. is not online on jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:41,263 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:55:41,270 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 02:55:41,272 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:55:41,280 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:41,280 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,280 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,281 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,282 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,282 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-08 02:55:41,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testRecoveryAndDoubleExecution 2023-02-08 02:55:41,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,284 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,285 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824941285"}]},"ts":"1675824941285"} 2023-02-08 02:55:41,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 02:55:41,286 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-08 02:55:41,386 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,386 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,386 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-08 02:55:41,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 02:55:41,388 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, UNASSIGN}] 2023-02-08 02:55:41,390 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, UNASSIGN 2023-02-08 02:55:41,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=becfd878a7a2594e71d99031c61388c1, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:41,391 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824941391"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824941391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824941391"}]},"ts":"1675824941391"} 2023-02-08 02:55:41,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:41,397 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, UNASSIGN 2023-02-08 02:55:41,398 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=45371de536b6594f3d1730c5e4920f27, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:41,398 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824941398"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824941398"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824941398"}]},"ts":"1675824941398"} 2023-02-08 02:55:41,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:41,529 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 506934cb-153f-48e7-8277-804b83b91344) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:41,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:41,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing becfd878a7a2594e71d99031c61388c1, disabling compactions & flushes 2023-02-08 02:55:41,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:41,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:41,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. after waiting 0 ms 2023-02-08 02:55:41,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:41,548 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-top] to archive 2023-02-08 02:55:41,550 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:41,553 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,556 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,559 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,561 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,564 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,566 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,568 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,570 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-top] to archive 2023-02-08 02:55:41,570 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:41,572 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,573 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,574 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,575 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,576 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,577 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,578 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:55:41,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:41,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1. 2023-02-08 02:55:41,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for becfd878a7a2594e71d99031c61388c1: Waiting for close lock at 1675824941547Running coprocessor pre-close hooks at 1675824941547Disabling compacts and flushes for region at 1675824941547Disabling writes for close at 1675824941548 (+1 ms)Writing region close event to WAL at 1675824941579 (+31 ms)Running coprocessor post-close hooks at 1675824941582 (+3 ms)Closed at 1675824941582 2023-02-08 02:55:41,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:41,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:41,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 45371de536b6594f3d1730c5e4920f27, disabling compactions & flushes 2023-02-08 02:55:41,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:41,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:41,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. after waiting 0 ms 2023-02-08 02:55:41,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:41,584 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=becfd878a7a2594e71d99031c61388c1, regionState=CLOSED 2023-02-08 02:55:41,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824941584"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824941584"}]},"ts":"1675824941584"} 2023-02-08 02:55:41,586 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73-bottom] to archive 2023-02-08 02:55:41,586 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:41,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-08 02:55:41,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure becfd878a7a2594e71d99031c61388c1, server=jenkins-hbase12.apache.org,37605,1675824833879 in 192 msec 2023-02-08 02:55:41,587 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=becfd878a7a2594e71d99031c61388c1, UNASSIGN in 199 msec 2023-02-08 02:55:41,588 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,588 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,588 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 02:55:41,589 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,590 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,590 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,591 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,592 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,593 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b-bottom] to archive 2023-02-08 02:55:41,594 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:41,594 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,595 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,596 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,597 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,598 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,599 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,599 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:55:41,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:41,603 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27. 2023-02-08 02:55:41,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 45371de536b6594f3d1730c5e4920f27: Waiting for close lock at 1675824941584Running coprocessor pre-close hooks at 1675824941584Disabling compacts and flushes for region at 1675824941584Disabling writes for close at 1675824941584Writing region close event to WAL at 1675824941600 (+16 ms)Running coprocessor post-close hooks at 1675824941603 (+3 ms)Closed at 1675824941603 2023-02-08 02:55:41,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:41,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=45371de536b6594f3d1730c5e4920f27, regionState=CLOSED 2023-02-08 02:55:41,605 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675824941605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824941605"}]},"ts":"1675824941605"} 2023-02-08 02:55:41,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-08 02:55:41,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 45371de536b6594f3d1730c5e4920f27, server=jenkins-hbase12.apache.org,37605,1675824833879 in 207 msec 2023-02-08 02:55:41,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-08 02:55:41,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=45371de536b6594f3d1730c5e4920f27, UNASSIGN in 219 msec 2023-02-08 02:55:41,608 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824941608"}]},"ts":"1675824941608"} 2023-02-08 02:55:41,609 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-08 02:55:41,753 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-08 02:55:41,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 472 msec 2023-02-08 02:55:41,889 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,889 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-08 02:55:41,891 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-08 02:55:41,891 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,891 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testRecoveryAndDoubleExecution 2023-02-08 02:55:41,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,894 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,894 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 02:55:41,900 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,900 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:41,900 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:41,903 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/recovered.edits] 2023-02-08 02:55:41,903 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/recovered.edits] 2023-02-08 02:55:41,903 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/recovered.edits] 2023-02-08 02:55:41,907 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/.regioninfo to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/.regioninfo 2023-02-08 02:55:41,907 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/.regioninfo to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/.regioninfo 2023-02-08 02:55:41,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/028a58f236824295b9a245455138a7a7 2023-02-08 02:55:41,908 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/140afc9e16894cb5984ed7fa0132e3ab 2023-02-08 02:55:41,909 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/313c6d3b051e4ccd953c2cb947a3ec38 2023-02-08 02:55:41,910 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,910 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/028a58f236824295b9a245455138a7a7.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,911 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/818e44c7840a4ddcb09e7a7de81f7ab3 2023-02-08 02:55:41,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,911 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/140afc9e16894cb5984ed7fa0132e3ab.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,912 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/9e501b9d143147d195d533a89be5d9d8 2023-02-08 02:55:41,912 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,913 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/313c6d3b051e4ccd953c2cb947a3ec38.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,913 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/ad5e815b62fa4eab943ef5992fc5fb14 2023-02-08 02:55:41,914 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,914 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/818e44c7840a4ddcb09e7a7de81f7ab3.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,914 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf1/c9e62bb01b4c411e82efb7fe1a258a73 2023-02-08 02:55:41,915 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,915 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/9e501b9d143147d195d533a89be5d9d8.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,916 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,917 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/ad5e815b62fa4eab943ef5992fc5fb14.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/6669fa63f61f4763a1b6d3e9a1447882 2023-02-08 02:55:41,921 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,921 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf1/c9e62bb01b4c411e82efb7fe1a258a73.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,922 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b 2023-02-08 02:55:41,923 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/792a8b2297e44bcfbf35e53022f531fb 2023-02-08 02:55:41,923 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,924 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/6669fa63f61f4763a1b6d3e9a1447882.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,925 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/8f34f56669ae4545a43229127e97d991 2023-02-08 02:55:41,925 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,925 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/77e2ff3cd5ab427bbc13973acc2d4f2b.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,926 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/9ff6cc273faf43d093bfc9e3818f7560 2023-02-08 02:55:41,926 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,927 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/792a8b2297e44bcfbf35e53022f531fb.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,928 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/c610ca09c4ac47f698875c5e4c839a65 2023-02-08 02:55:41,928 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,929 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/cf2/d019281617c14eadbe4422c06c4efa45 2023-02-08 02:55:41,929 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/8f34f56669ae4545a43229127e97d991.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,929 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,930 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/9ff6cc273faf43d093bfc9e3818f7560.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,931 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,932 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/c610ca09c4ac47f698875c5e4c839a65.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,932 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c/recovered.edits/85.seqid 2023-02-08 02:55:41,932 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/becfd878a7a2594e71d99031c61388c1/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,932 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,933 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/45371de536b6594f3d1730c5e4920f27/cf2/d019281617c14eadbe4422c06c4efa45.760d9a76a1f3e3afbc3d42d93be1360c 2023-02-08 02:55:41,935 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/e84e80ec31c945978fa8fd4f566b05af to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf1/e84e80ec31c945978fa8fd4f566b05af 2023-02-08 02:55:41,935 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/0e23874ad3ca4f10955778e4d186185a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf1/0e23874ad3ca4f10955778e4d186185a 2023-02-08 02:55:41,936 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c8b130ed70c04a9e9f62d1dfe92cb7dc to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/cf2/c8b130ed70c04a9e9f62d1dfe92cb7dc 2023-02-08 02:55:41,937 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/36877a5631d14c5a85beab71f21bd3fa to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/cf2/36877a5631d14c5a85beab71f21bd3fa 2023-02-08 02:55:41,938 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1/recovered.edits/90.seqid 2023-02-08 02:55:41,939 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27/recovered.edits/90.seqid 2023-02-08 02:55:41,939 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/becfd878a7a2594e71d99031c61388c1 2023-02-08 02:55:41,939 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testRecoveryAndDoubleExecution/45371de536b6594f3d1730c5e4920f27 2023-02-08 02:55:41,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-08 02:55:41,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,942 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-08 02:55:41,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-08 02:55:41,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-08 02:55:41,945 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824941945"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:41,945 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824941945"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:41,945 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824941945"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:41,946 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:55:41,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 760d9a76a1f3e3afbc3d42d93be1360c, NAME => 'testRecoveryAndDoubleExecution,,1675824934493.760d9a76a1f3e3afbc3d42d93be1360c.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 45371de536b6594f3d1730c5e4920f27, NAME => 'testRecoveryAndDoubleExecution,,1675824937159.45371de536b6594f3d1730c5e4920f27.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => becfd878a7a2594e71d99031c61388c1, NAME => 'testRecoveryAndDoubleExecution,41,1675824937159.becfd878a7a2594e71d99031c61388c1.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 02:55:41,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-08 02:55:41,946 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824941946"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:41,947 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-08 02:55:41,955 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-08 02:55:41,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 63 msec 2023-02-08 02:55:41,996 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:41,996 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:41,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-08 02:55:41,997 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-08 02:55:42,018 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=373 (was 366) - Thread LEAK? -, OpenFileDescriptor=839 (was 742) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=220 (was 213) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=3561 (was 3602) 2023-02-08 02:55:42,029 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=373, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=220, ProcessCount=172, AvailableMemoryMB=3561 2023-02-08 02:55:42,030 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:42,030 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:42,030 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:42,039 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:55:42,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:55:42,040 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,042 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:55:42,042 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:55:42,044 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,044 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 empty. 2023-02-08 02:55:42,045 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,045 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 02:55:42,057 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-08 02:55:42,058 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => d6ee21ef6032b22aa5f4649a38667365, NAME => 'testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing d6ee21ef6032b22aa5f4649a38667365, disabling compactions & flushes 2023-02-08 02:55:42,068 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. after waiting 0 ms 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,068 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,068 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for d6ee21ef6032b22aa5f4649a38667365: Waiting for close lock at 1675824942068Disabling compacts and flushes for region at 1675824942068Disabling writes for close at 1675824942068Writing region close event to WAL at 1675824942068Closed at 1675824942068 2023-02-08 02:55:42,071 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:55:42,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824942072"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824942072"}]},"ts":"1675824942072"} 2023-02-08 02:55:42,073 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:55:42,074 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:55:42,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824942074"}]},"ts":"1675824942074"} 2023-02-08 02:55:42,076 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-08 02:55:42,092 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:55:42,093 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:55:42,093 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:55:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:55:42,094 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:55:42,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, ASSIGN}] 2023-02-08 02:55:42,096 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, ASSIGN 2023-02-08 02:55:42,098 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:42,248 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:55:42,251 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=d6ee21ef6032b22aa5f4649a38667365, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:42,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824942251"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824942251"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824942251"}]},"ts":"1675824942251"} 2023-02-08 02:55:42,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure d6ee21ef6032b22aa5f4649a38667365, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:42,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d6ee21ef6032b22aa5f4649a38667365, NAME => 'testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:55:42,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,417 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,419 DEBUG [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1 2023-02-08 02:55:42,419 DEBUG [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1 2023-02-08 02:55:42,419 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6ee21ef6032b22aa5f4649a38667365 columnFamilyName cf1 2023-02-08 02:55:42,420 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] regionserver.HStore(310): Store=d6ee21ef6032b22aa5f4649a38667365/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:42,420 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,422 DEBUG [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2 2023-02-08 02:55:42,422 DEBUG [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2 2023-02-08 02:55:42,422 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6ee21ef6032b22aa5f4649a38667365 columnFamilyName cf2 2023-02-08 02:55:42,423 INFO [StoreOpener-d6ee21ef6032b22aa5f4649a38667365-1] regionserver.HStore(310): Store=d6ee21ef6032b22aa5f4649a38667365/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:42,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:42,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:42,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:55:42,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened d6ee21ef6032b22aa5f4649a38667365; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71500450, jitterRate=0.06543973088264465}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:42,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for d6ee21ef6032b22aa5f4649a38667365: Running coprocessor pre-open hook at 1675824942415Writing region info on filesystem at 1675824942415Initializing all the Stores at 1675824942416 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824942416Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824942417 (+1 ms)Cleaning up temporary data from old regions at 1675824942425 (+8 ms)Cleaning up detritus from prior splits at 1675824942426 (+1 ms)Running coprocessor post-open hooks at 1675824942431 (+5 ms)Region opened successfully at 1675824942431 2023-02-08 02:55:42,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365., pid=127, masterSystemTime=1675824942409 2023-02-08 02:55:42,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=d6ee21ef6032b22aa5f4649a38667365, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:42,434 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824942434"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824942434"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824942434"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824942434"}]},"ts":"1675824942434"} 2023-02-08 02:55:42,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-08 02:55:42,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure d6ee21ef6032b22aa5f4649a38667365, server=jenkins-hbase12.apache.org,37605,1675824833879 in 181 msec 2023-02-08 02:55:42,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-08 02:55:42,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, ASSIGN in 349 msec 2023-02-08 02:55:42,445 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:55:42,446 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824942446"}]},"ts":"1675824942446"} 2023-02-08 02:55:42,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-08 02:55:42,502 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:55:42,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 462 msec 2023-02-08 02:55:42,527 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 8382d232-5ed5-4e55-9828-0c5294970b5d) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:42,552 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:42,552 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:42,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:42,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,555 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:42,555 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:42,555 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:42,555 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:42,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:42,575 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:42,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:42,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:42,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:42,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:42,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:42,576 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:42,576 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:42,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:42,577 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:42,578 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:42,578 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:42,578 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:42,579 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:55:42,596 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,618 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,618 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,618 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:42,619 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:42,619 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:42,991 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/941ed2aaff344756a5ecc5863a55892e 2023-02-08 02:55:43,412 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/90669526ba5141a7a015ad91ffbfbe22 2023-02-08 02:55:43,426 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/941ed2aaff344756a5ecc5863a55892e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e 2023-02-08 02:55:43,432 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:43,432 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/90669526ba5141a7a015ad91ffbfbe22 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22 2023-02-08 02:55:43,436 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:43,436 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 857ms, sequenceid=6, compaction requested=false 2023-02-08 02:55:43,436 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:43,437 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:43,437 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:43,437 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:43,437 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:43,437 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,437 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:43,437 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:43,481 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,481 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,481 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,481 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:43,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:43,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dae780f[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:43,486 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:43,486 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,492 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,492 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,492 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:43,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,492 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,493 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:43,492 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,492 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:43,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:43,502 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,502 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,502 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,513 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,524 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:43,535 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:43,535 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:43,535 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:43,535 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:43,535 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,535 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:43,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,545 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,546 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,547 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,550 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,551 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,552 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,599 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,599 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,599 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:43,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:43,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,598 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,599 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:43,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,673 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:55:43,700 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:43,700 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:43,700 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:43,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:43,712 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:43,712 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:43,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:43,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:43,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,714 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:43,714 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:43,714 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:43,714 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:43,714 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,714 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,723 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,723 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,723 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,723 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:43,724 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:43,724 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,724 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:43,724 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:43,724 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:43,724 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:43,725 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:43,725 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:43,725 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:43,725 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:43,733 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,733 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,734 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,734 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,734 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:43,734 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:43,739 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 2023-02-08 02:55:43,753 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/a5b9b6181b654a53872e1068e536d8a6 2023-02-08 02:55:43,756 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 2023-02-08 02:55:43,760 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:43,760 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/a5b9b6181b654a53872e1068e536d8a6 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6 2023-02-08 02:55:43,764 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:43,765 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 40ms, sequenceid=19, compaction requested=false 2023-02-08 02:55:43,765 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:43,765 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:43,765 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:43,765 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:43,765 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,765 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:43,765 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:43,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,775 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,776 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,776 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66e7431c[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:43,777 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:43,777 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:43,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:43,797 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:43,807 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,807 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,807 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,807 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,818 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,818 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,818 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,828 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:43,828 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,828 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:43,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:43,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:43,839 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:43,839 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:43,839 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:43,839 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,839 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,850 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:43,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:43,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,850 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:43,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,852 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,853 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:43,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,856 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:43,856 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,857 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,870 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:43,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:43,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,870 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:43,871 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:43,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:43,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:43,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:43,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:43,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:43,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,972 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:43,972 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:43,972 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:43,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:43,991 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:43,992 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:43,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:43,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:43,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,994 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:43,994 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:43,995 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:43,995 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:43,995 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:43,995 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,007 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,008 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,009 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:44,009 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:44,009 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:44,009 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:44,009 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:44,009 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:44,017 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,054 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,054 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:44,054 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,055 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,055 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:44,417 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/364a817303e64056873bcf2610f46e7a 2023-02-08 02:55:44,439 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/1a6627a3784c4a0c924555a7df6824d8 2023-02-08 02:55:44,442 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/364a817303e64056873bcf2610f46e7a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a 2023-02-08 02:55:44,445 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:44,445 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/1a6627a3784c4a0c924555a7df6824d8 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8 2023-02-08 02:55:44,449 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:44,450 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 441ms, sequenceid=32, compaction requested=true 2023-02-08 02:55:44,450 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:44,450 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:44,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:44,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:44,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:44,450 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:44,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,491 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:44,492 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,492 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:44,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:44,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:44,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:44,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e799ea3[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:44,498 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:44,498 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,511 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:44,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:44,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:44,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:44,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:44,513 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:44,512 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,513 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:44,513 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:44,514 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:44,514 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:44,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:44,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:44,523 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:44,534 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:44,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:44,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:44,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:44,534 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:44,534 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:44,535 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:44,535 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:44,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:44,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:44,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:44,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:44,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:44,547 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:44,547 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:44,547 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:44,547 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,547 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:44,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:44,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,555 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,556 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:44,558 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,559 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,559 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,574 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:44,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:44,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:44,574 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:44,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,575 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:44,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:44,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,576 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,676 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:44,676 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:44,676 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:44,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:44,692 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:44,692 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:44,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:44,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:44,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,694 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:44,694 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,694 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:44,695 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:44,695 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,695 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:44,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:44,711 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,712 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,712 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,712 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:44,713 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:44,713 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:44,713 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:44,713 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:44,723 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:45,125 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/95108325aad14db2b125605e0e3b462a 2023-02-08 02:55:45,136 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/f5b01e60a2514fa08ba9994c0c60afb9 2023-02-08 02:55:45,139 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/95108325aad14db2b125605e0e3b462a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a 2023-02-08 02:55:45,142 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:45,143 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/f5b01e60a2514fa08ba9994c0c60afb9 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9 2023-02-08 02:55:45,147 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:45,148 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 435ms, sequenceid=45, compaction requested=true 2023-02-08 02:55:45,148 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:45,148 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:45,148 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:45,148 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:45,148 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,148 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:45,148 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:45,154 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,154 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:45,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:45,155 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,155 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:45,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:45,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ae5d8a2[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:45,157 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:45,158 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:45,165 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,165 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:45,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,165 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:45,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:45,166 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:45,175 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:45,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,175 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:45,175 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:45,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:45,175 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:45,175 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:45,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:45,186 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:45,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:45,186 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:45,186 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:45,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:45,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:45,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:45,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:45,190 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:45,190 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:45,190 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:45,190 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,190 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,273 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:45,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,274 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:45,275 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:45,276 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,277 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,277 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,421 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,421 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,421 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:45,421 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:45,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,421 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,420 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:45,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,421 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,421 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:45,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,522 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:45,523 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:45,523 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:45,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:45,535 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:45,535 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:45,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:45,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:45,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,537 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:45,537 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:45,537 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:45,538 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:45,538 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,538 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,564 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,564 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,564 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:45,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,564 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,565 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:45,565 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:45,566 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,567 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:45,567 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:45,567 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:45,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:45,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:45,570 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:45,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,570 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:45,570 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:45,571 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:45,571 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:45,570 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:45,581 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,592 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,592 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:45,592 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,592 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:45,592 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:45,984 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/8a09a037f3ca42be92a251eb54d9d64e 2023-02-08 02:55:45,994 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/b7711dd10e36445b9f88fa05e305ffc9 2023-02-08 02:55:45,997 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/8a09a037f3ca42be92a251eb54d9d64e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e 2023-02-08 02:55:46,000 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:46,001 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/b7711dd10e36445b9f88fa05e305ffc9 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9 2023-02-08 02:55:46,004 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:46,004 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 434ms, sequenceid=58, compaction requested=true 2023-02-08 02:55:46,004 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:46,004 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,004 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:46,005 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:46,005 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,005 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:46,005 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:46,034 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,034 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,035 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,035 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:46,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24f5adf1[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:46,036 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:46,036 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,129 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,129 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,129 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,130 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,130 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,131 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:46,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,130 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,131 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,131 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,131 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:46,131 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:46,131 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,132 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,132 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,144 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,154 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,154 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,154 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,154 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:46,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:46,163 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:46,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:46,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:46,163 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,163 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,176 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,176 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,177 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,180 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,181 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,181 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,196 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:46,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:46,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,197 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:46,197 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:46,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,196 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,198 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,297 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:46,298 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:46,298 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:46,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:46,312 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:46,312 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:46,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:46,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:46,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,315 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:46,315 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:46,315 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-08 02:55:46,315 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-08 02:55:46,315 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,315 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,322 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,322 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,322 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,322 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,323 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,323 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,323 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:46,324 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:46,324 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:46,324 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,324 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:46,324 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:46,325 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:46,325 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:46,325 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,325 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. started... 2023-02-08 02:55:46,325 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:46,335 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/cf4b251aab9442a5950863c01719a85e 2023-02-08 02:55:46,348 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/3a98b0be6dd44a42ae178b57df4952f3 2023-02-08 02:55:46,352 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/cf4b251aab9442a5950863c01719a85e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e 2023-02-08 02:55:46,356 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:46,357 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/3a98b0be6dd44a42ae178b57df4952f3 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3 2023-02-08 02:55:46,358 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,360 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:46,361 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 36ms, sequenceid=71, compaction requested=true 2023-02-08 02:55:46,361 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d6ee21ef6032b22aa5f4649a38667365: 2023-02-08 02:55:46,361 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,361 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:46,361 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:46,361 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,361 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-08 02:55:46,361 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-08 02:55:46,375 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,375 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:46,375 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,386 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,386 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:46,386 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,386 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,386 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-08 02:55:46,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78d6e5d0[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:46,389 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-08 02:55:46,389 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,396 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,396 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-08 02:55:46,397 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,407 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,417 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-08 02:55:46,417 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,417 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-08 02:55:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:46,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-08 02:55:46,420 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-08 02:55:46,420 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:46,420 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:46,420 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,420 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,428 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:46,429 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,429 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,431 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:46,431 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-08 02:55:46,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,438 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:46,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:46,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,439 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:46,440 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,440 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,440 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-08 02:55:46,532 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 506934cb-153f-48e7-8277-804b83b91344) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:46,539 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:46,540 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:46,540 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:46,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:46,555 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,556 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d6ee21ef6032b22aa5f4649a38667365, daughterA=74c59e60f1f126326325705ffb27b875, daughterB=53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:46,557 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d6ee21ef6032b22aa5f4649a38667365, daughterA=74c59e60f1f126326325705ffb27b875, daughterB=53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:46,557 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d6ee21ef6032b22aa5f4649a38667365, daughterA=74c59e60f1f126326325705ffb27b875, daughterB=53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:46,557 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d6ee21ef6032b22aa5f4649a38667365, daughterA=74c59e60f1f126326325705ffb27b875, daughterB=53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:46,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, UNASSIGN}] 2023-02-08 02:55:46,560 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, UNASSIGN 2023-02-08 02:55:46,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=d6ee21ef6032b22aa5f4649a38667365, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:46,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824946560"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824946560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824946560"}]},"ts":"1675824946560"} 2023-02-08 02:55:46,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure d6ee21ef6032b22aa5f4649a38667365, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:46,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing d6ee21ef6032b22aa5f4649a38667365, disabling compactions & flushes 2023-02-08 02:55:46,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. after waiting 0 ms 2023-02-08 02:55:46,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing d6ee21ef6032b22aa5f4649a38667365 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:55:46,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 2023-02-08 02:55:46,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/c12c5443714b486492c4f36706fb5eaa 2023-02-08 02:55:46,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 2023-02-08 02:55:46,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:46,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.tmp/cf2/c12c5443714b486492c4f36706fb5eaa as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa 2023-02-08 02:55:46,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:46,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 46ms, sequenceid=82, compaction requested=true 2023-02-08 02:55:46,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:55:46,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:46,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. 2023-02-08 02:55:46,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for d6ee21ef6032b22aa5f4649a38667365: Waiting for close lock at 1675824946714Running coprocessor pre-close hooks at 1675824946714Disabling compacts and flushes for region at 1675824946714Disabling writes for close at 1675824946715 (+1 ms)Obtaining lock to block concurrent updates at 1675824946715Preparing flush snapshotting stores in d6ee21ef6032b22aa5f4649a38667365 at 1675824946715Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824946715Flushing stores of testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. at 1675824946717 (+2 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf1: creating writer at 1675824946721 (+4 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf1: appending metadata at 1675824946725 (+4 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf1: closing flushed file at 1675824946725Flushing d6ee21ef6032b22aa5f4649a38667365/cf2: creating writer at 1675824946740 (+15 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf2: appending metadata at 1675824946742 (+2 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf2: closing flushed file at 1675824946742Flushing d6ee21ef6032b22aa5f4649a38667365/cf1: reopening flushed file at 1675824946752 (+10 ms)Flushing d6ee21ef6032b22aa5f4649a38667365/cf2: reopening flushed file at 1675824946756 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d6ee21ef6032b22aa5f4649a38667365 in 46ms, sequenceid=82, compaction requested=true at 1675824946761 (+5 ms)Writing region close event to WAL at 1675824946762 (+1 ms)Running coprocessor post-close hooks at 1675824946766 (+4 ms)Closed at 1675824946767 (+1 ms) 2023-02-08 02:55:46,768 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=d6ee21ef6032b22aa5f4649a38667365, regionState=CLOSED 2023-02-08 02:55:46,770 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824946769"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824946769"}]},"ts":"1675824946769"} 2023-02-08 02:55:46,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-08 02:55:46,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure d6ee21ef6032b22aa5f4649a38667365, server=jenkins-hbase12.apache.org,37605,1675824833879 in 210 msec 2023-02-08 02:55:46,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-08 02:55:46,774 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=d6ee21ef6032b22aa5f4649a38667365, UNASSIGN in 214 msec 2023-02-08 02:55:46,787 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=d6ee21ef6032b22aa5f4649a38667365, threads=14 2023-02-08 02:55:46,787 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,787 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,788 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,788 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,793 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,793 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,793 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,793 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,793 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,794 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,794 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,794 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,794 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,794 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,796 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8, top=false 2023-02-08 02:55:46,800 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6, top=false 2023-02-08 02:55:46,800 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9, top=false 2023-02-08 02:55:46,801 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22, top=false 2023-02-08 02:55:46,801 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a, top=false 2023-02-08 02:55:46,803 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a, top=false 2023-02-08 02:55:46,803 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3, top=false 2023-02-08 02:55:46,803 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e, top=false 2023-02-08 02:55:46,808 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66, top=false 2023-02-08 02:55:46,809 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9, top=false 2023-02-08 02:55:46,811 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1, top=false 2023-02-08 02:55:46,812 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e, top=false 2023-02-08 02:55:46,813 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e, top=false 2023-02-08 02:55:46,814 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa, top=false 2023-02-08 02:55:46,816 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,823 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,827 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,829 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,833 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,833 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,833 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,835 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,841 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,841 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,841 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,842 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,844 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,845 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,849 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,850 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,850 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,850 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,851 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,850 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8 for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,851 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/.splits/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e for child: 74c59e60f1f126326325705ffb27b875, parent: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,852 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,854 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,855 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,856 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3 for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,856 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,857 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,864 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e for region: d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:46,864 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region d6ee21ef6032b22aa5f4649a38667365 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-08 02:55:46,912 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:46,914 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:46,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675824946916"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675824946916"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675824946916"}]},"ts":"1675824946916"} 2023-02-08 02:55:46,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824946916"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824946916"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824946916"}]},"ts":"1675824946916"} 2023-02-08 02:55:46,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824946916"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824946916"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824946916"}]},"ts":"1675824946916"} 2023-02-08 02:55:46,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, ASSIGN}] 2023-02-08 02:55:46,923 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, ASSIGN 2023-02-08 02:55:46,923 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:46,924 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, ASSIGN 2023-02-08 02:55:46,924 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:47,074 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:55:47,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=53d692ca3a8ddca3f02bf504403854a2, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,075 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947075"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824947075"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824947075"}]},"ts":"1675824947075"} 2023-02-08 02:55:47,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 53d692ca3a8ddca3f02bf504403854a2, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:47,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=74c59e60f1f126326325705ffb27b875, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947079"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824947079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824947079"}]},"ts":"1675824947079"} 2023-02-08 02:55:47,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 74c59e60f1f126326325705ffb27b875, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:47,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 53d692ca3a8ddca3f02bf504403854a2, NAME => 'testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.', STARTKEY => '71', ENDKEY => ''} 2023-02-08 02:55:47,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:47,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:47,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,235 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,237 DEBUG [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf1 2023-02-08 02:55:47,237 DEBUG [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf1 2023-02-08 02:55:47,238 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53d692ca3a8ddca3f02bf504403854a2 columnFamilyName cf1 2023-02-08 02:55:47,238 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] regionserver.HStore(310): Store=53d692ca3a8ddca3f02bf504403854a2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:47,239 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,240 DEBUG [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf2 2023-02-08 02:55:47,240 DEBUG [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf2 2023-02-08 02:55:47,241 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53d692ca3a8ddca3f02bf504403854a2 columnFamilyName cf2 2023-02-08 02:55:47,241 INFO [StoreOpener-53d692ca3a8ddca3f02bf504403854a2-1] regionserver.HStore(310): Store=53d692ca3a8ddca3f02bf504403854a2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:47,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:47,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 53d692ca3a8ddca3f02bf504403854a2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65024293, jitterRate=-0.031062528491020203}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:47,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 53d692ca3a8ddca3f02bf504403854a2: Running coprocessor pre-open hook at 1675824947234Writing region info on filesystem at 1675824947234Initializing all the Stores at 1675824947235 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824947235Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824947235Cleaning up temporary data from old regions at 1675824947244 (+9 ms)Cleaning up detritus from prior splits at 1675824947244Running coprocessor post-open hooks at 1675824947247 (+3 ms)Region opened successfully at 1675824947247 2023-02-08 02:55:47,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2., pid=133, masterSystemTime=1675824947228 2023-02-08 02:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 74c59e60f1f126326325705ffb27b875, NAME => 'testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.', STARTKEY => '', ENDKEY => '71'} 2023-02-08 02:55:47,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=53d692ca3a8ddca3f02bf504403854a2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:47,250 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947250"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824947250"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824947250"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824947250"}]},"ts":"1675824947250"} 2023-02-08 02:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:47,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,252 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,253 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1 2023-02-08 02:55:47,253 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1 2023-02-08 02:55:47,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-08 02:55:47,254 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74c59e60f1f126326325705ffb27b875 columnFamilyName cf1 2023-02-08 02:55:47,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 53d692ca3a8ddca3f02bf504403854a2, server=jenkins-hbase12.apache.org,37605,1675824833879 in 175 msec 2023-02-08 02:55:47,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, ASSIGN in 332 msec 2023-02-08 02:55:47,260 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1 2023-02-08 02:55:47,263 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a 2023-02-08 02:55:47,266 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e 2023-02-08 02:55:47,268 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e 2023-02-08 02:55:47,271 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a 2023-02-08 02:55:47,274 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 2023-02-08 02:55:47,277 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e 2023-02-08 02:55:47,277 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(310): Store=74c59e60f1f126326325705ffb27b875/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:47,277 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,278 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2 2023-02-08 02:55:47,278 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2 2023-02-08 02:55:47,279 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74c59e60f1f126326325705ffb27b875 columnFamilyName cf2 2023-02-08 02:55:47,285 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8 2023-02-08 02:55:47,289 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3 2023-02-08 02:55:47,294 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22 2023-02-08 02:55:47,297 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6 2023-02-08 02:55:47,300 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9 2023-02-08 02:55:47,304 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa 2023-02-08 02:55:47,307 DEBUG [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9 2023-02-08 02:55:47,307 INFO [StoreOpener-74c59e60f1f126326325705ffb27b875-1] regionserver.HStore(310): Store=74c59e60f1f126326325705ffb27b875/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:47,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:47,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 74c59e60f1f126326325705ffb27b875; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61494130, jitterRate=-0.08366605639457703}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:47,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 74c59e60f1f126326325705ffb27b875: Running coprocessor pre-open hook at 1675824947251Writing region info on filesystem at 1675824947251Initializing all the Stores at 1675824947251Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824947252 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824947252Cleaning up temporary data from old regions at 1675824947310 (+58 ms)Cleaning up detritus from prior splits at 1675824947310Running coprocessor post-open hooks at 1675824947313 (+3 ms)Region opened successfully at 1675824947313 2023-02-08 02:55:47,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875., pid=134, masterSystemTime=1675824947228 2023-02-08 02:55:47,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:47,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): 74c59e60f1f126326325705ffb27b875/cf1 is initiating minor compaction (all files) 2023-02-08 02:55:47,316 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): 74c59e60f1f126326325705ffb27b875/cf2 is initiating minor compaction (all files) 2023-02-08 02:55:47,316 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 74c59e60f1f126326325705ffb27b875/cf1 in testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,316 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 74c59e60f1f126326325705ffb27b875/cf2 in testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,316 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/.tmp, totalSize=34.9 K 2023-02-08 02:55:47,316 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/.tmp, totalSize=34.9 K 2023-02-08 02:55:47,317 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824942544 2023-02-08 02:55:47,317 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824942544 2023-02-08 02:55:47,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=74c59e60f1f126326325705ffb27b875, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,318 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947318"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824947318"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824947318"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824947318"}]},"ts":"1675824947318"} 2023-02-08 02:55:47,318 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824943703 2023-02-08 02:55:47,318 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824943703 2023-02-08 02:55:47,319 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824943976 2023-02-08 02:55:47,319 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824943976 2023-02-08 02:55:47,319 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824944679 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824944679 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824945525 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824945525 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824946299 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824946299 2023-02-08 02:55:47,320 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824946543 2023-02-08 02:55:47,321 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824946543 2023-02-08 02:55:47,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-08 02:55:47,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 74c59e60f1f126326325705ffb27b875, server=jenkins-hbase12.apache.org,37605,1675824833879 in 236 msec 2023-02-08 02:55:47,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-08 02:55:47,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, ASSIGN in 401 msec 2023-02-08 02:55:47,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=d6ee21ef6032b22aa5f4649a38667365, daughterA=74c59e60f1f126326325705ffb27b875, daughterB=53d692ca3a8ddca3f02bf504403854a2 in 770 msec 2023-02-08 02:55:47,341 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 74c59e60f1f126326325705ffb27b875#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:47,341 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74c59e60f1f126326325705ffb27b875#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:47,349 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/.tmp/cf1/51971882f6d844938bc7842e28a3cef3 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/51971882f6d844938bc7842e28a3cef3 2023-02-08 02:55:47,349 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/.tmp/cf2/6aa37245eaf445bfac98fac0f65d17de as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/6aa37245eaf445bfac98fac0f65d17de 2023-02-08 02:55:47,353 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 74c59e60f1f126326325705ffb27b875/cf2 of 74c59e60f1f126326325705ffb27b875 into 6aa37245eaf445bfac98fac0f65d17de(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:47,353 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 74c59e60f1f126326325705ffb27b875: 2023-02-08 02:55:47,354 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875., storeName=74c59e60f1f126326325705ffb27b875/cf2, priority=9, startTime=1675824947315; duration=0sec 2023-02-08 02:55:47,354 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:47,355 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 74c59e60f1f126326325705ffb27b875/cf1 of 74c59e60f1f126326325705ffb27b875 into 51971882f6d844938bc7842e28a3cef3(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:47,355 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 74c59e60f1f126326325705ffb27b875: 2023-02-08 02:55:47,355 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875., storeName=74c59e60f1f126326325705ffb27b875/cf1, priority=9, startTime=1675824947314; duration=0sec 2023-02-08 02:55:47,355 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:47,529 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 8382d232-5ed5-4e55-9828-0c5294970b5d) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:47,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37605] ipc.CallRunner(144): callId: 993 service: ClientService methodName: Scan size: 157 connection: 136.243.104.168:32972 deadline: 1675825007557, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365. is not online on jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,669 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-08 02:55:47,673 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:47,673 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,673 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,674 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,675 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,675 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,675 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,678 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,678 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 02:55:47,679 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824947679"}]},"ts":"1675824947679"} 2023-02-08 02:55:47,680 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-08 02:55:47,701 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-08 02:55:47,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, UNASSIGN}] 2023-02-08 02:55:47,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, UNASSIGN 2023-02-08 02:55:47,705 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=53d692ca3a8ddca3f02bf504403854a2, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,705 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947705"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824947705"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824947705"}]},"ts":"1675824947705"} 2023-02-08 02:55:47,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 53d692ca3a8ddca3f02bf504403854a2, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:47,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, UNASSIGN 2023-02-08 02:55:47,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=74c59e60f1f126326325705ffb27b875, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:47,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947712"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824947712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824947712"}]},"ts":"1675824947712"} 2023-02-08 02:55:47,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 74c59e60f1f126326325705ffb27b875, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:47,779 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,780 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 02:55:47,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 53d692ca3a8ddca3f02bf504403854a2, disabling compactions & flushes 2023-02-08 02:55:47,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. after waiting 0 ms 2023-02-08 02:55:47,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,865 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-08 02:55:47,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:47,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2. 2023-02-08 02:55:47,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 53d692ca3a8ddca3f02bf504403854a2: Waiting for close lock at 1675824947861Running coprocessor pre-close hooks at 1675824947861Disabling compacts and flushes for region at 1675824947861Disabling writes for close at 1675824947862 (+1 ms)Writing region close event to WAL at 1675824947862Running coprocessor post-close hooks at 1675824947866 (+4 ms)Closed at 1675824947866 2023-02-08 02:55:47,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:47,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 74c59e60f1f126326325705ffb27b875, disabling compactions & flushes 2023-02-08 02:55:47,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. after waiting 0 ms 2023-02-08 02:55:47,868 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=53d692ca3a8ddca3f02bf504403854a2, regionState=CLOSED 2023-02-08 02:55:47,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,868 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824947867"}]},"ts":"1675824947867"} 2023-02-08 02:55:47,868 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66] to archive 2023-02-08 02:55:47,869 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:47,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-08 02:55:47,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 53d692ca3a8ddca3f02bf504403854a2, server=jenkins-hbase12.apache.org,37605,1675824833879 in 162 msec 2023-02-08 02:55:47,872 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-941ed2aaff344756a5ecc5863a55892e 2023-02-08 02:55:47,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=53d692ca3a8ddca3f02bf504403854a2, UNASSIGN in 168 msec 2023-02-08 02:55:47,873 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-079a9475d9cd4566a0bbd6b6eaae4ab1 2023-02-08 02:55:47,875 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-364a817303e64056873bcf2610f46e7a 2023-02-08 02:55:47,876 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-95108325aad14db2b125605e0e3b462a 2023-02-08 02:55:47,878 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-8a09a037f3ca42be92a251eb54d9d64e 2023-02-08 02:55:47,880 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-cf4b251aab9442a5950863c01719a85e 2023-02-08 02:55:47,881 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 2023-02-08 02:55:47,883 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa] to archive 2023-02-08 02:55:47,884 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:47,885 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-90669526ba5141a7a015ad91ffbfbe22 2023-02-08 02:55:47,886 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-a5b9b6181b654a53872e1068e536d8a6 2023-02-08 02:55:47,887 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-1a6627a3784c4a0c924555a7df6824d8 2023-02-08 02:55:47,888 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-f5b01e60a2514fa08ba9994c0c60afb9 2023-02-08 02:55:47,890 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-b7711dd10e36445b9f88fa05e305ffc9 2023-02-08 02:55:47,891 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-3a98b0be6dd44a42ae178b57df4952f3 2023-02-08 02:55:47,891 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/testSplitTableRegionEmptyDaughter=d6ee21ef6032b22aa5f4649a38667365-c12c5443714b486492c4f36706fb5eaa 2023-02-08 02:55:47,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:55:47,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:47,895 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875. 2023-02-08 02:55:47,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 74c59e60f1f126326325705ffb27b875: Waiting for close lock at 1675824947867Running coprocessor pre-close hooks at 1675824947867Disabling compacts and flushes for region at 1675824947867Disabling writes for close at 1675824947868 (+1 ms)Writing region close event to WAL at 1675824947892 (+24 ms)Running coprocessor post-close hooks at 1675824947895 (+3 ms)Closed at 1675824947895 2023-02-08 02:55:47,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:47,897 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=74c59e60f1f126326325705ffb27b875, regionState=CLOSED 2023-02-08 02:55:47,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675824947897"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824947897"}]},"ts":"1675824947897"} 2023-02-08 02:55:47,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-08 02:55:47,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 74c59e60f1f126326325705ffb27b875, server=jenkins-hbase12.apache.org,37605,1675824833879 in 186 msec 2023-02-08 02:55:47,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-08 02:55:47,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=74c59e60f1f126326325705ffb27b875, UNASSIGN in 198 msec 2023-02-08 02:55:47,903 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824947903"}]},"ts":"1675824947903"} 2023-02-08 02:55:47,904 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-08 02:55:47,912 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-08 02:55:47,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 237 msec 2023-02-08 02:55:47,982 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,983 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-08 02:55:47,985 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-08 02:55:47,985 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,985 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,991 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,991 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:47,992 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:47,992 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:47,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 02:55:48,000 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:48,000 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:48,000 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:48,002 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/recovered.edits] 2023-02-08 02:55:48,002 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/recovered.edits] 2023-02-08 02:55:48,002 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/recovered.edits] 2023-02-08 02:55:48,005 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/079a9475d9cd4566a0bbd6b6eaae4ab1 2023-02-08 02:55:48,005 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/51971882f6d844938bc7842e28a3cef3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf1/51971882f6d844938bc7842e28a3cef3 2023-02-08 02:55:48,006 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/364a817303e64056873bcf2610f46e7a 2023-02-08 02:55:48,007 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/recovered.edits/88.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2/recovered.edits/88.seqid 2023-02-08 02:55:48,007 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/8a09a037f3ca42be92a251eb54d9d64e 2023-02-08 02:55:48,008 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/53d692ca3a8ddca3f02bf504403854a2 2023-02-08 02:55:48,008 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/6aa37245eaf445bfac98fac0f65d17de to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/cf2/6aa37245eaf445bfac98fac0f65d17de 2023-02-08 02:55:48,009 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/941ed2aaff344756a5ecc5863a55892e 2023-02-08 02:55:48,010 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/95108325aad14db2b125605e0e3b462a 2023-02-08 02:55:48,010 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875/recovered.edits/90.seqid 2023-02-08 02:55:48,010 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/74c59e60f1f126326325705ffb27b875 2023-02-08 02:55:48,011 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/9ba2ec6ec41b4ee1a3d5f41cc6ef1f66 2023-02-08 02:55:48,012 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf1/cf4b251aab9442a5950863c01719a85e 2023-02-08 02:55:48,013 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/1a6627a3784c4a0c924555a7df6824d8 2023-02-08 02:55:48,014 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/3a98b0be6dd44a42ae178b57df4952f3 2023-02-08 02:55:48,015 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/90669526ba5141a7a015ad91ffbfbe22 2023-02-08 02:55:48,016 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/a5b9b6181b654a53872e1068e536d8a6 2023-02-08 02:55:48,016 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/b7711dd10e36445b9f88fa05e305ffc9 2023-02-08 02:55:48,017 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/c12c5443714b486492c4f36706fb5eaa 2023-02-08 02:55:48,018 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/cf2/f5b01e60a2514fa08ba9994c0c60afb9 2023-02-08 02:55:48,020 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365/recovered.edits/85.seqid 2023-02-08 02:55:48,020 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegionEmptyDaughter/d6ee21ef6032b22aa5f4649a38667365 2023-02-08 02:55:48,020 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-08 02:55:48,022 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:48,024 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-08 02:55:48,026 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-08 02:55:48,027 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:48,027 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-08 02:55:48,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824948027"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:48,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824948027"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:48,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824948027"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:48,028 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:55:48,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d6ee21ef6032b22aa5f4649a38667365, NAME => 'testSplitTableRegionEmptyDaughter,,1675824942040.d6ee21ef6032b22aa5f4649a38667365.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 74c59e60f1f126326325705ffb27b875, NAME => 'testSplitTableRegionEmptyDaughter,,1675824946555.74c59e60f1f126326325705ffb27b875.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 53d692ca3a8ddca3f02bf504403854a2, NAME => 'testSplitTableRegionEmptyDaughter,71,1675824946555.53d692ca3a8ddca3f02bf504403854a2.', STARTKEY => '71', ENDKEY => ''}] 2023-02-08 02:55:48,028 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-08 02:55:48,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824948028"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:48,029 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-08 02:55:48,039 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-08 02:55:48,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 52 msec 2023-02-08 02:55:48,095 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:48,095 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:48,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-08 02:55:48,096 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-08 02:55:48,118 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=372 (was 373), OpenFileDescriptor=839 (was 839), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=224 (was 220) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=3462 (was 3561) 2023-02-08 02:55:48,136 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=372, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=172, AvailableMemoryMB=3461 2023-02-08 02:55:48,137 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:48,137 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:48,137 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:48,150 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-08 02:55:48,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(492): Client=jenkins//136.243.104.168 set balanceSwitch=false 2023-02-08 02:55:48,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-08 02:55:48,153 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 02:55:48,154 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 02:55:48,155 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,155 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 empty. 2023-02-08 02:55:48,156 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 02:55:48,172 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-08 02:55:48,173 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3483493d94d3b49573f86ca8f1b07c01, NAME => 'testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp 2023-02-08 02:55:48,583 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:48,584 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 3483493d94d3b49573f86ca8f1b07c01, disabling compactions & flushes 2023-02-08 02:55:48,584 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,584 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,584 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. after waiting 0 ms 2023-02-08 02:55:48,584 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,584 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,584 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 3483493d94d3b49573f86ca8f1b07c01: Waiting for close lock at 1675824948584Disabling compacts and flushes for region at 1675824948584Disabling writes for close at 1675824948584Writing region close event to WAL at 1675824948584Closed at 1675824948584 2023-02-08 02:55:48,590 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 02:55:48,592 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824948592"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824948592"}]},"ts":"1675824948592"} 2023-02-08 02:55:48,594 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 02:55:48,595 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 02:55:48,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824948595"}]},"ts":"1675824948595"} 2023-02-08 02:55:48,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-08 02:55:48,617 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase12.apache.org=0} racks are {/default-rack=0} 2023-02-08 02:55:48,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-08 02:55:48,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-08 02:55:48,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-08 02:55:48,618 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-08 02:55:48,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, ASSIGN}] 2023-02-08 02:55:48,620 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, ASSIGN 2023-02-08 02:55:48,620 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, ASSIGN; state=OFFLINE, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:48,770 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-08 02:55:48,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=3483493d94d3b49573f86ca8f1b07c01, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:48,773 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824948772"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824948772"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824948772"}]},"ts":"1675824948772"} 2023-02-08 02:55:48,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 3483493d94d3b49573f86ca8f1b07c01, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:48,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3483493d94d3b49573f86ca8f1b07c01, NAME => 'testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.', STARTKEY => '', ENDKEY => ''} 2023-02-08 02:55:48,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:48,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:48,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,940 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,943 DEBUG [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1 2023-02-08 02:55:48,944 DEBUG [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1 2023-02-08 02:55:48,944 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3483493d94d3b49573f86ca8f1b07c01 columnFamilyName cf1 2023-02-08 02:55:48,945 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] regionserver.HStore(310): Store=3483493d94d3b49573f86ca8f1b07c01/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:48,945 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,946 DEBUG [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2 2023-02-08 02:55:48,946 DEBUG [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2 2023-02-08 02:55:48,946 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3483493d94d3b49573f86ca8f1b07c01 columnFamilyName cf2 2023-02-08 02:55:48,947 INFO [StoreOpener-3483493d94d3b49573f86ca8f1b07c01-1] regionserver.HStore(310): Store=3483493d94d3b49573f86ca8f1b07c01/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:48,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,948 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:48,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:48,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 02:55:48,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 3483493d94d3b49573f86ca8f1b07c01; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63542487, jitterRate=-0.05314315855503082}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:48,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 3483493d94d3b49573f86ca8f1b07c01: Running coprocessor pre-open hook at 1675824948937Writing region info on filesystem at 1675824948937Initializing all the Stores at 1675824948939 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824948939Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824948940 (+1 ms)Cleaning up temporary data from old regions at 1675824948948 (+8 ms)Cleaning up detritus from prior splits at 1675824948948Running coprocessor post-open hooks at 1675824948950 (+2 ms)Region opened successfully at 1675824948950 2023-02-08 02:55:48,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01., pid=143, masterSystemTime=1675824948929 2023-02-08 02:55:48,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:48,974 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=3483493d94d3b49573f86ca8f1b07c01, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:48,974 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824948974"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824948974"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824948974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824948974"}]},"ts":"1675824948974"} 2023-02-08 02:55:48,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-08 02:55:48,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 3483493d94d3b49573f86ca8f1b07c01, server=jenkins-hbase12.apache.org,37605,1675824833879 in 198 msec 2023-02-08 02:55:48,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-08 02:55:48,977 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, ASSIGN in 358 msec 2023-02-08 02:55:48,977 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 02:55:48,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824948977"}]},"ts":"1675824948977"} 2023-02-08 02:55:48,978 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-08 02:55:49,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 02:55:49,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 878 msec 2023-02-08 02:55:49,158 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:49,158 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:49,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:49,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:49,160 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:49,160 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,160 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:49,160 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:49,160 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,160 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,175 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,176 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:49,177 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:49,177 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:49,177 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:49,178 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-08 02:55:49,185 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,191 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e 2023-02-08 02:55:49,196 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,196 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,196 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,196 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,196 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,206 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/03b9ce359ce5430fbc4d34d9379f3dab 2023-02-08 02:55:49,210 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e 2023-02-08 02:55:49,213 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:49,214 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/03b9ce359ce5430fbc4d34d9379f3dab as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab 2023-02-08 02:55:49,217 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:49,218 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 40ms, sequenceid=6, compaction requested=false 2023-02-08 02:55:49,218 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-08 02:55:49,218 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:49,218 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,218 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,227 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,227 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,228 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,228 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:49,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ad8e0ef[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:49,230 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:49,230 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,269 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,269 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,269 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,270 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,270 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,270 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,270 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,270 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:49,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,270 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,271 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,271 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,271 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:49,271 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,271 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:49,291 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,291 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,291 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,291 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,312 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:49,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:49,317 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:49,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:49,317 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,330 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,331 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,331 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,334 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,335 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,335 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:49,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:49,350 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,351 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,351 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:49,351 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,353 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,353 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,452 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:49,452 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:49,452 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:49,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:49,474 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:49,475 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:49,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:49,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:49,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:49,478 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:49,478 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,478 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:49,478 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:49,479 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,479 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,491 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,491 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,493 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,494 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,494 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:49,494 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:49,494 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:49,494 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:49,494 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:49,501 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,512 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 2023-02-08 02:55:49,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,512 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,512 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,521 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/88f35dc1ef7b4656b4fdbc4080847668 2023-02-08 02:55:49,524 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 2023-02-08 02:55:49,527 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:49,528 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/88f35dc1ef7b4656b4fdbc4080847668 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668 2023-02-08 02:55:49,531 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668, entries=10, sequenceid=19, filesize=5.0 K 2023-02-08 02:55:49,532 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 38ms, sequenceid=19, compaction requested=false 2023-02-08 02:55:49,532 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:49,532 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:49,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:49,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:49,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,532 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,543 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,543 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,544 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,544 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:49,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e0c17de[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:49,546 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:49,546 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,554 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,554 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:49,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:49,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:49,554 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:49,564 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,564 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,564 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,575 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,586 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:49,586 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,586 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:49,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:49,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:49,592 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:49,592 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:49,592 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:49,592 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:49,593 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:49,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,604 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:49,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,603 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,605 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,606 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:49,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,609 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:49,609 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:49,610 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:49,651 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:49,651 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:49,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:49,651 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:49,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,752 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:49,752 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:49,752 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:49,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:49,771 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:49,771 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:49,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:49,774 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:49,774 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,774 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:49,774 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:49,775 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,775 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,796 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:49,796 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:49,797 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:49,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,799 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:49,799 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,799 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,800 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,800 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:49,801 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:49,801 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:49,801 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:49,801 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:49,813 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,813 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,813 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:49,823 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,824 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:49,824 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,215 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/a394bc9df218479d828df011e8208744 2023-02-08 02:55:50,232 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/28124afe734849d78fbea306b6486b2c 2023-02-08 02:55:50,236 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/a394bc9df218479d828df011e8208744 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744 2023-02-08 02:55:50,240 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:50,241 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/28124afe734849d78fbea306b6486b2c as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c 2023-02-08 02:55:50,245 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-08 02:55:50,246 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 445ms, sequenceid=32, compaction requested=true 2023-02-08 02:55:50,246 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:50,246 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:50,246 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:50,246 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:50,246 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,246 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,246 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,259 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,260 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,260 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:50,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79fa9cb[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:50,264 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:50,264 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,276 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:50,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,276 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:50,277 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:50,287 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,287 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,287 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,297 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,297 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,297 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,297 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,297 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,297 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,298 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:50,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:50,301 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:50,301 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:50,301 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:50,301 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:50,301 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,312 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,313 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,314 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,314 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,317 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,318 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,387 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,387 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,387 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:50,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:50,387 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,388 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:50,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,389 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,389 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:50,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,389 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,490 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:50,490 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:50,490 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:50,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:50,507 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:50,507 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:50,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:50,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:50,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:50,511 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:50,511 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,511 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:50,511 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:50,512 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,512 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,533 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,534 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,534 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,536 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,536 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,536 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,538 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,537 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,538 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,537 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:50,538 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:50,538 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,538 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,538 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,538 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:50,547 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/07be6d207ea84107b9438314224b9ba1 2023-02-08 02:55:50,557 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/019f17c219184d5d82906805131e5a7d 2023-02-08 02:55:50,561 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/07be6d207ea84107b9438314224b9ba1 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1 2023-02-08 02:55:50,565 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:50,566 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/019f17c219184d5d82906805131e5a7d as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d 2023-02-08 02:55:50,569 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-08 02:55:50,570 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 32ms, sequenceid=45, compaction requested=true 2023-02-08 02:55:50,570 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:50,570 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:50,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:50,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:50,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,570 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,629 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,655 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,655 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,656 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,665 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,665 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,665 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,666 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,666 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,666 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:50,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48a8348d[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:50,668 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:50,668 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,680 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,681 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:50,680 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:50,681 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:50,691 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,691 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,691 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,727 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:50,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:50,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:50,740 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:50,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:50,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:50,740 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:50,740 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,761 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:50,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,761 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,762 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:50,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,763 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,763 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:50,763 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:50,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,775 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:50,775 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:50,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,775 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:50,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,776 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:50,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:50,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,776 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:50,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:50,876 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:50,876 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:50,876 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:50,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:50,885 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:50,885 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:50,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:50,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:50,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:50,887 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:50,887 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,887 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:50,887 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:50,888 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,888 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,897 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:50,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,898 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:50,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:50,899 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,899 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,899 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,899 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,900 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,900 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,900 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,900 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:50,900 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:50,911 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,912 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:50,922 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/c8a273b718c94f7f94f39b97d8f81414 2023-02-08 02:55:50,926 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 2023-02-08 02:55:50,929 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:50,929 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/c8a273b718c94f7f94f39b97d8f81414 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414 2023-02-08 02:55:50,933 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414, entries=10, sequenceid=58, filesize=5.0 K 2023-02-08 02:55:50,934 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 34ms, sequenceid=58, compaction requested=true 2023-02-08 02:55:50,934 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:50,934 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:50,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:50,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:50,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:50,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:50,934 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:51,010 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,010 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,011 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,011 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:51,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:51,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61bd1237[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:51,018 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:51,019 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,049 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:51,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,049 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,049 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:51,050 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:51,059 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,059 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,059 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,070 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,070 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,070 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,070 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,070 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,071 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,071 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:51,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:51,083 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:51,083 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:51,083 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:51,084 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:51,084 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:51,097 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,097 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,097 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,097 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,098 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,099 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,098 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,100 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,100 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,104 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,104 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,104 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,118 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:51,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:51,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,119 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:51,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,119 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,119 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:51,119 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,120 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,220 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:51,220 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:51,220 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:51,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:51,242 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:51,243 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:51,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(933): Client=jenkins//136.243.104.168 procedure request for: flush-table-proc 2023-02-08 02:55:51,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-08 02:55:51,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-08 02:55:51,246 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-08 02:55:51,246 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:51,246 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-08 02:55:51,246 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-08 02:55:51,247 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,247 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,259 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,259 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,260 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 02:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:51,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:51,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:51,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-08 02:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:51,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:51,263 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,46783,1675824833788' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:51,262 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,36075,1675824833838' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:51,263 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,263 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 02:55:51,263 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 02:55:51,263 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. started... 2023-02-08 02:55:51,263 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-08 02:55:51,275 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/d56f4934dfcb444f948258a1bf772b80 2023-02-08 02:55:51,289 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/32e5a593f41b4d9d998c09cca7533dc2 2023-02-08 02:55:51,294 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/d56f4934dfcb444f948258a1bf772b80 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80 2023-02-08 02:55:51,297 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:51,298 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/32e5a593f41b4d9d998c09cca7533dc2 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2 2023-02-08 02:55:51,298 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,298 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,299 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,299 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,299 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:51,299 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:51,301 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-08 02:55:51,302 INFO [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 39ms, sequenceid=71, compaction requested=true 2023-02-08 02:55:51,302 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3483493d94d3b49573f86ca8f1b07c01: 2023-02-08 02:55:51,302 DEBUG [rs(jenkins-hbase12.apache.org,37605,1675824833879)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,302 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 02:55:51,302 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 02:55:51,302 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,302 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-08 02:55:51,302 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-08 02:55:51,329 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,329 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,329 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,329 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 02:55:51,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase12.apache.org,37605,1675824833879' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-08 02:55:51,332 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-08 02:55:51,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@68384ed1[Count = 0] remaining members to acquire global barrier 2023-02-08 02:55:51,332 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,37605,1675824833879' in zk 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,350 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,350 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-08 02:55:51,350 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,46783,1675824833788' in zk 2023-02-08 02:55:51,351 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-08 02:55:51,351 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase12.apache.org,36075,1675824833838' in zk 2023-02-08 02:55:51,371 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,371 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,371 DEBUG [member: 'jenkins-hbase12.apache.org,37605,1675824833879' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,46783,1675824833788' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,401 DEBUG [member: 'jenkins-hbase12.apache.org,36075,1675824833838' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-08 02:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase12.apache.org,37605,1675824833879': 2023-02-08 02:55:51,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase12.apache.org,37605,1675824833879' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-08 02:55:51,403 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-08 02:55:51,403 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 02:55:51,403 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 02:55:51,404 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-08 02:55:51,404 INFO [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 02:55:51,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,428 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,428 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 02:55:51,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,428 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 02:55:51,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 02:55:51,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,430 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 02:55:51,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-08 02:55:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 02:55:51,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-08 02:55:51,449 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 02:55:51,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,449 DEBUG [(jenkins-hbase12.apache.org,39013,1675824832471)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 02:55:51,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-08 02:55:51,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 02:55:51,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-08 02:55:51,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:51,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 02:55:51,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-08 02:55:51,450 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-08 02:55:51,550 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 02:55:51,550 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:51,550 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:51,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 02:55:51,558 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,563 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=3483493d94d3b49573f86ca8f1b07c01, daughterA=c839d1c01914d7e4669d3cac724aca6b, daughterB=18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:51,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=3483493d94d3b49573f86ca8f1b07c01, daughterA=c839d1c01914d7e4669d3cac724aca6b, daughterB=18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:51,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=3483493d94d3b49573f86ca8f1b07c01, daughterA=c839d1c01914d7e4669d3cac724aca6b, daughterB=18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:51,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=3483493d94d3b49573f86ca8f1b07c01, daughterA=c839d1c01914d7e4669d3cac724aca6b, daughterB=18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:51,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, UNASSIGN}] 2023-02-08 02:55:51,566 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, UNASSIGN 2023-02-08 02:55:51,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=3483493d94d3b49573f86ca8f1b07c01, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:51,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824951569"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824951569"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824951569"}]},"ts":"1675824951569"} 2023-02-08 02:55:51,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 3483493d94d3b49573f86ca8f1b07c01, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:51,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 3483493d94d3b49573f86ca8f1b07c01, disabling compactions & flushes 2023-02-08 02:55:51,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. after waiting 0 ms 2023-02-08 02:55:51,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 3483493d94d3b49573f86ca8f1b07c01 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-08 02:55:51,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/69f942e90abb4c42888d6c03d0e212cf 2023-02-08 02:55:51,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/8e5cc008adc24c53b6ef74cf48dda992 2023-02-08 02:55:51,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf1/69f942e90abb4c42888d6c03d0e212cf as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf 2023-02-08 02:55:51,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:51,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/.tmp/cf2/8e5cc008adc24c53b6ef74cf48dda992 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992 2023-02-08 02:55:51,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992, entries=8, sequenceid=82, filesize=5.0 K 2023-02-08 02:55:51,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 48ms, sequenceid=82, compaction requested=true 2023-02-08 02:55:51,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-08 02:55:51,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:51,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. 2023-02-08 02:55:51,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 3483493d94d3b49573f86ca8f1b07c01: Waiting for close lock at 1675824951723Running coprocessor pre-close hooks at 1675824951723Disabling compacts and flushes for region at 1675824951723Disabling writes for close at 1675824951723Obtaining lock to block concurrent updates at 1675824951724 (+1 ms)Preparing flush snapshotting stores in 3483493d94d3b49573f86ca8f1b07c01 at 1675824951724Finished memstore snapshotting testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675824951724Flushing stores of testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. at 1675824951725 (+1 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf1: creating writer at 1675824951727 (+2 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf1: appending metadata at 1675824951730 (+3 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf1: closing flushed file at 1675824951730Flushing 3483493d94d3b49573f86ca8f1b07c01/cf2: creating writer at 1675824951745 (+15 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf2: appending metadata at 1675824951747 (+2 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf2: closing flushed file at 1675824951747Flushing 3483493d94d3b49573f86ca8f1b07c01/cf1: reopening flushed file at 1675824951761 (+14 ms)Flushing 3483493d94d3b49573f86ca8f1b07c01/cf2: reopening flushed file at 1675824951765 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3483493d94d3b49573f86ca8f1b07c01 in 48ms, sequenceid=82, compaction requested=true at 1675824951771 (+6 ms)Writing region close event to WAL at 1675824951773 (+2 ms)Running coprocessor post-close hooks at 1675824951776 (+3 ms)Closed at 1675824951777 (+1 ms) 2023-02-08 02:55:51,778 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,778 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=3483493d94d3b49573f86ca8f1b07c01, regionState=CLOSED 2023-02-08 02:55:51,779 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824951778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824951778"}]},"ts":"1675824951778"} 2023-02-08 02:55:51,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-08 02:55:51,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 3483493d94d3b49573f86ca8f1b07c01, server=jenkins-hbase12.apache.org,37605,1675824833879 in 210 msec 2023-02-08 02:55:51,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-08 02:55:51,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3483493d94d3b49573f86ca8f1b07c01, UNASSIGN in 216 msec 2023-02-08 02:55:51,799 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=3483493d94d3b49573f86ca8f1b07c01, threads=14 2023-02-08 02:55:51,799 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,799 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,800 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,801 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,882 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,889 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,917 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,921 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,926 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,926 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,945 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,949 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,950 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,951 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,951 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,951 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,951 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,952 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2 for region: 3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:51,952 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 3483493d94d3b49573f86ca8f1b07c01 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-08 02:55:51,988 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:51,990 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-08 02:55:51,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675824951991"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675824951991"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675824951991"}]},"ts":"1675824951991"} 2023-02-08 02:55:51,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824951991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824951991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824951991"}]},"ts":"1675824951991"} 2023-02-08 02:55:51,991 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824951991"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824951991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824951991"}]},"ts":"1675824951991"} 2023-02-08 02:55:51,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, ASSIGN}] 2023-02-08 02:55:51,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, ASSIGN 2023-02-08 02:55:51,995 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:51,995 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, ASSIGN 2023-02-08 02:55:51,996 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase12.apache.org,37605,1675824833879; forceNewPlan=false, retain=false 2023-02-08 02:55:52,145 INFO [jenkins-hbase12:39013] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-08 02:55:52,147 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=18cda76e13e82f572bb3aeb083809cfc, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,147 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952147"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824952147"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824952147"}]},"ts":"1675824952147"} 2023-02-08 02:55:52,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 18cda76e13e82f572bb3aeb083809cfc, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:52,153 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c839d1c01914d7e4669d3cac724aca6b, regionState=OPENING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,153 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952153"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824952153"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824952153"}]},"ts":"1675824952153"} 2023-02-08 02:55:52,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure c839d1c01914d7e4669d3cac724aca6b, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:52,270 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 02:55:52,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c839d1c01914d7e4669d3cac724aca6b, NAME => 'testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.', STARTKEY => '', ENDKEY => '41'} 2023-02-08 02:55:52,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:52,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:52,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,306 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,307 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1 2023-02-08 02:55:52,307 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1 2023-02-08 02:55:52,307 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c839d1c01914d7e4669d3cac724aca6b columnFamilyName cf1 2023-02-08 02:55:52,318 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-bottom 2023-02-08 02:55:52,321 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-bottom 2023-02-08 02:55:52,324 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-bottom 2023-02-08 02:55:52,328 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-bottom 2023-02-08 02:55:52,331 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-bottom 2023-02-08 02:55:52,335 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-bottom 2023-02-08 02:55:52,338 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-bottom 2023-02-08 02:55:52,338 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(310): Store=c839d1c01914d7e4669d3cac724aca6b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:52,338 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,339 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2 2023-02-08 02:55:52,339 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2 2023-02-08 02:55:52,340 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c839d1c01914d7e4669d3cac724aca6b columnFamilyName cf2 2023-02-08 02:55:52,350 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-bottom 2023-02-08 02:55:52,354 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-bottom 2023-02-08 02:55:52,358 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-bottom 2023-02-08 02:55:52,361 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-bottom 2023-02-08 02:55:52,364 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-bottom 2023-02-08 02:55:52,367 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-bottom 2023-02-08 02:55:52,370 DEBUG [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-bottom 2023-02-08 02:55:52,370 INFO [StoreOpener-c839d1c01914d7e4669d3cac724aca6b-1] regionserver.HStore(310): Store=c839d1c01914d7e4669d3cac724aca6b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:52,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:52,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:52,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened c839d1c01914d7e4669d3cac724aca6b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64476020, jitterRate=-0.03923243284225464}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:52,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for c839d1c01914d7e4669d3cac724aca6b: Running coprocessor pre-open hook at 1675824952305Writing region info on filesystem at 1675824952305Initializing all the Stores at 1675824952305Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824952305Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824952306 (+1 ms)Cleaning up temporary data from old regions at 1675824952372 (+66 ms)Cleaning up detritus from prior splits at 1675824952372Running coprocessor post-open hooks at 1675824952374 (+2 ms)Region opened successfully at 1675824952374 2023-02-08 02:55:52,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b., pid=150, masterSystemTime=1675824952302 2023-02-08 02:55:52,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:52,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:52,375 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:52,375 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,376 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1912): c839d1c01914d7e4669d3cac724aca6b/cf1 is initiating minor compaction (all files) 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c839d1c01914d7e4669d3cac724aca6b/cf1 in testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,376 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): c839d1c01914d7e4669d3cac724aca6b/cf2 is initiating minor compaction (all files) 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c839d1c01914d7e4669d3cac724aca6b/cf2 in testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/.tmp, totalSize=34.9 K 2023-02-08 02:55:52,376 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-bottom] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/.tmp, totalSize=34.9 K 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824949154 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675824949154 2023-02-08 02:55:52,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824949456 2023-02-08 02:55:52,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,377 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c839d1c01914d7e4669d3cac724aca6b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675824949456 2023-02-08 02:55:52,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 18cda76e13e82f572bb3aeb083809cfc, NAME => 'testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.', STARTKEY => '41', ENDKEY => ''} 2023-02-08 02:55:52,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952377"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824952377"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824952377"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824952377"}]},"ts":"1675824952377"} 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824949755 2023-02-08 02:55:52,377 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675824949755 2023-02-08 02:55:52,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824950491 2023-02-08 02:55:52,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675824950491 2023-02-08 02:55:52,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 02:55:52,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7890): checking encryption for 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(7893): checking classloading for 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824950877 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675824950877 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824951224 2023-02-08 02:55:52,378 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675824951224 2023-02-08 02:55:52,379 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,379 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] compactions.Compactor(207): Compacting 69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824951551 2023-02-08 02:55:52,379 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675824951551 2023-02-08 02:55:52,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-08 02:55:52,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure c839d1c01914d7e4669d3cac724aca6b, server=jenkins-hbase12.apache.org,37605,1675824833879 in 223 msec 2023-02-08 02:55:52,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, ASSIGN in 387 msec 2023-02-08 02:55:52,387 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1 2023-02-08 02:55:52,387 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1 2023-02-08 02:55:52,387 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18cda76e13e82f572bb3aeb083809cfc columnFamilyName cf1 2023-02-08 02:55:52,395 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] throttle.PressureAwareThroughputController(145): c839d1c01914d7e4669d3cac724aca6b#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:52,398 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): c839d1c01914d7e4669d3cac724aca6b#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:52,401 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-top 2023-02-08 02:55:52,406 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-top 2023-02-08 02:55:52,410 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-top 2023-02-08 02:55:52,411 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/.tmp/cf2/8ac6b1b53a714a21ab6c5eb926df5526 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8ac6b1b53a714a21ab6c5eb926df5526 2023-02-08 02:55:52,412 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-top 2023-02-08 02:55:52,414 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-top 2023-02-08 02:55:52,415 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c839d1c01914d7e4669d3cac724aca6b/cf2 of c839d1c01914d7e4669d3cac724aca6b into 8ac6b1b53a714a21ab6c5eb926df5526(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:52,415 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c839d1c01914d7e4669d3cac724aca6b: 2023-02-08 02:55:52,415 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b., storeName=c839d1c01914d7e4669d3cac724aca6b/cf2, priority=9, startTime=1675824952375; duration=0sec 2023-02-08 02:55:52,415 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:52,417 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-top 2023-02-08 02:55:52,420 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-top 2023-02-08 02:55:52,420 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(310): Store=18cda76e13e82f572bb3aeb083809cfc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:52,420 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,421 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2 2023-02-08 02:55:52,421 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2 2023-02-08 02:55:52,421 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18cda76e13e82f572bb3aeb083809cfc columnFamilyName cf2 2023-02-08 02:55:52,432 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-top 2023-02-08 02:55:52,435 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-top 2023-02-08 02:55:52,438 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-top 2023-02-08 02:55:52,441 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-top 2023-02-08 02:55:52,443 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-top 2023-02-08 02:55:52,446 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-top 2023-02-08 02:55:52,451 DEBUG [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-top 2023-02-08 02:55:52,451 INFO [StoreOpener-18cda76e13e82f572bb3aeb083809cfc-1] regionserver.HStore(310): Store=18cda76e13e82f572bb3aeb083809cfc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 02:55:52,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 02:55:52,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1054): writing seq id for 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1071): Opened 18cda76e13e82f572bb3aeb083809cfc; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68039552, jitterRate=0.013868331909179688}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 02:55:52,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(964): Region open journal for 18cda76e13e82f572bb3aeb083809cfc: Running coprocessor pre-open hook at 1675824952378Writing region info on filesystem at 1675824952378Initializing all the Stores at 1675824952379 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824952379Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675824952379Cleaning up temporary data from old regions at 1675824952452 (+73 ms)Cleaning up detritus from prior splits at 1675824952453 (+1 ms)Running coprocessor post-open hooks at 1675824952454 (+1 ms)Region opened successfully at 1675824952454 2023-02-08 02:55:52,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc., pid=149, masterSystemTime=1675824952302 2023-02-08 02:55:52,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:52,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:52,456 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:52,456 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,456 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): 18cda76e13e82f572bb3aeb083809cfc/cf1 is initiating minor compaction (all files) 2023-02-08 02:55:52,456 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 18cda76e13e82f572bb3aeb083809cfc/cf1 in testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,457 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/.tmp, totalSize=34.9 K 2023-02-08 02:55:52,457 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824949154 2023-02-08 02:55:52,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase12:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,458 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824949456 2023-02-08 02:55:52,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=18cda76e13e82f572bb3aeb083809cfc, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952458"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1675824952458"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675824952458"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675824952458"}]},"ts":"1675824952458"} 2023-02-08 02:55:52,458 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824949755 2023-02-08 02:55:52,459 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824950491 2023-02-08 02:55:52,459 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824950877 2023-02-08 02:55:52,460 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824951224 2023-02-08 02:55:52,460 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824951551 2023-02-08 02:55:52,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-08 02:55:52,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 18cda76e13e82f572bb3aeb083809cfc, server=jenkins-hbase12.apache.org,37605,1675824833879 in 310 msec 2023-02-08 02:55:52,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-08 02:55:52,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, ASSIGN in 466 msec 2023-02-08 02:55:52,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=3483493d94d3b49573f86ca8f1b07c01, daughterA=c839d1c01914d7e4669d3cac724aca6b, daughterB=18cda76e13e82f572bb3aeb083809cfc in 904 msec 2023-02-08 02:55:52,468 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 18cda76e13e82f572bb3aeb083809cfc#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:52,480 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/.tmp/cf1/d235df02936041d39dd0420711abd684 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d235df02936041d39dd0420711abd684 2023-02-08 02:55:52,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37605] ipc.CallRunner(144): callId: 1122 service: ClientService methodName: Scan size: 143 connection: 136.243.104.168:32972 deadline: 1675825012629, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01. is not online on jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,630 ERROR [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 8382d232-5ed5-4e55-9828-0c5294970b5d) service to localhost.localdomain/127.0.0.1:36267 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-08 02:55:52,631 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18cda76e13e82f572bb3aeb083809cfc/cf1 of 18cda76e13e82f572bb3aeb083809cfc into d235df02936041d39dd0420711abd684(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:52,631 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 18cda76e13e82f572bb3aeb083809cfc: 2023-02-08 02:55:52,631 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc., storeName=18cda76e13e82f572bb3aeb083809cfc/cf1, priority=9, startTime=1675824952455; duration=0sec 2023-02-08 02:55:52,631 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 02:55:52,631 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-08 02:55:52,631 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,631 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1912): 18cda76e13e82f572bb3aeb083809cfc/cf2 is initiating minor compaction (all files) 2023-02-08 02:55:52,631 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 18cda76e13e82f572bb3aeb083809cfc/cf2 in testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,632 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-top] into tmpdir=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/.tmp, totalSize=34.9 K 2023-02-08 02:55:52,632 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675824949154 2023-02-08 02:55:52,632 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675824949456 2023-02-08 02:55:52,633 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675824949755 2023-02-08 02:55:52,633 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675824950491 2023-02-08 02:55:52,634 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675824950877 2023-02-08 02:55:52,634 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675824951224 2023-02-08 02:55:52,634 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] compactions.Compactor(207): Compacting 8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675824951551 2023-02-08 02:55:52,644 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] throttle.PressureAwareThroughputController(145): 18cda76e13e82f572bb3aeb083809cfc#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 02:55:52,656 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/.tmp/cf2/047b347bb0a645ffa074829582fa23e6 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/047b347bb0a645ffa074829582fa23e6 2023-02-08 02:55:52,661 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18cda76e13e82f572bb3aeb083809cfc/cf2 of 18cda76e13e82f572bb3aeb083809cfc into 047b347bb0a645ffa074829582fa23e6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:52,661 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 18cda76e13e82f572bb3aeb083809cfc: 2023-02-08 02:55:52,661 INFO [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc., storeName=18cda76e13e82f572bb3aeb083809cfc/cf2, priority=9, startTime=1675824952456; duration=0sec 2023-02-08 02:55:52,661 DEBUG [RS:2;jenkins-hbase12:37605-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:52,739 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:55:52,747 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-08 02:55:52,749 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-08 02:55:52,771 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-08 02:55:52,771 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:52,771 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:52,772 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:52,772 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:52,772 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-08 02:55:52,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$11(2405): Client=jenkins//136.243.104.168 disable testSplitTableRegion 2023-02-08 02:55:52,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-08 02:55:52,775 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:52,775 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:52,776 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824952776"}]},"ts":"1675824952776"} 2023-02-08 02:55:52,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 02:55:52,777 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-08 02:55:52,809 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/.tmp/cf1/28d512ac2b99449b952e8124af3fe4d8 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/28d512ac2b99449b952e8124af3fe4d8 2023-02-08 02:55:52,813 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c839d1c01914d7e4669d3cac724aca6b/cf1 of c839d1c01914d7e4669d3cac724aca6b into 28d512ac2b99449b952e8124af3fe4d8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 02:55:52,813 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c839d1c01914d7e4669d3cac724aca6b: 2023-02-08 02:55:52,813 INFO [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b., storeName=c839d1c01914d7e4669d3cac724aca6b/cf1, priority=9, startTime=1675824952374; duration=0sec 2023-02-08 02:55:52,814 DEBUG [RS:2;jenkins-hbase12:37605-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 02:55:52,827 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-08 02:55:52,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, UNASSIGN}] 2023-02-08 02:55:52,831 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, UNASSIGN 2023-02-08 02:55:52,832 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=18cda76e13e82f572bb3aeb083809cfc, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,832 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952832"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824952832"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824952832"}]},"ts":"1675824952832"} 2023-02-08 02:55:52,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 18cda76e13e82f572bb3aeb083809cfc, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:52,836 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, UNASSIGN 2023-02-08 02:55:52,837 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c839d1c01914d7e4669d3cac724aca6b, regionState=CLOSING, regionLocation=jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:52,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824952837"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1675824952837"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675824952837"}]},"ts":"1675824952837"} 2023-02-08 02:55:52,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure c839d1c01914d7e4669d3cac724aca6b, server=jenkins-hbase12.apache.org,37605,1675824833879}] 2023-02-08 02:55:52,876 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:52,877 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:52,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 02:55:52,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:52,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 18cda76e13e82f572bb3aeb083809cfc, disabling compactions & flushes 2023-02-08 02:55:52,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. after waiting 0 ms 2023-02-08 02:55:52,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:52,987 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-top] to archive 2023-02-08 02:55:52,988 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:52,990 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,992 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,993 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,994 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,996 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,997 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:52,999 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,000 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-top, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-top] to archive 2023-02-08 02:55:53,001 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:53,003 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,004 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,006 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,008 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,009 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,010 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,012 DEBUG [StoreCloser-testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,017 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:55:53,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:53,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc. 2023-02-08 02:55:53,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 18cda76e13e82f572bb3aeb083809cfc: Waiting for close lock at 1675824952986Running coprocessor pre-close hooks at 1675824952986Disabling compacts and flushes for region at 1675824952986Disabling writes for close at 1675824952986Writing region close event to WAL at 1675824953013 (+27 ms)Running coprocessor post-close hooks at 1675824953018 (+5 ms)Closed at 1675824953018 2023-02-08 02:55:53,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed 18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:53,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(109): Close c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing c839d1c01914d7e4669d3cac724aca6b, disabling compactions & flushes 2023-02-08 02:55:53,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:53,020 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=18cda76e13e82f572bb3aeb083809cfc, regionState=CLOSED 2023-02-08 02:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. after waiting 0 ms 2023-02-08 02:55:53,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:53,020 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824953020"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824953020"}]},"ts":"1675824953020"} 2023-02-08 02:55:53,022 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf-bottom] to archive 2023-02-08 02:55:53,023 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:53,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-08 02:55:53,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 18cda76e13e82f572bb3aeb083809cfc, server=jenkins-hbase12.apache.org,37605,1675824833879 in 188 msec 2023-02-08 02:55:53,025 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=18cda76e13e82f572bb3aeb083809cfc, UNASSIGN in 195 msec 2023-02-08 02:55:53,027 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/f34c1c97f4b5460ea6db3931b2d32dd9.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,028 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/a394bc9df218479d828df011e8208744.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,030 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/07be6d207ea84107b9438314224b9ba1.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,031 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/14078b16fc1a4bd5b05ec3c32135e0e4.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,033 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/d56f4934dfcb444f948258a1bf772b80.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,034 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/69f942e90abb4c42888d6c03d0e212cf.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,036 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2-bottom, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01->hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992-bottom] to archive 2023-02-08 02:55:53,037 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 02:55:53,039 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/03b9ce359ce5430fbc4d34d9379f3dab.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,040 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/88f35dc1ef7b4656b4fdbc4080847668.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,042 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/28124afe734849d78fbea306b6486b2c.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,043 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/019f17c219184d5d82906805131e5a7d.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,045 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/c8a273b718c94f7f94f39b97d8f81414.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,046 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/32e5a593f41b4d9d998c09cca7533dc2.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,047 DEBUG [StoreCloser-testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8e5cc008adc24c53b6ef74cf48dda992.3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-08 02:55:53,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:53,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b. 2023-02-08 02:55:53,051 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for c839d1c01914d7e4669d3cac724aca6b: Waiting for close lock at 1675824953020Running coprocessor pre-close hooks at 1675824953020Disabling compacts and flushes for region at 1675824953020Disabling writes for close at 1675824953020Writing region close event to WAL at 1675824953048 (+28 ms)Running coprocessor post-close hooks at 1675824953051 (+3 ms)Closed at 1675824953051 2023-02-08 02:55:53,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.UnassignRegionHandler(147): Closed c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:53,053 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c839d1c01914d7e4669d3cac724aca6b, regionState=CLOSED 2023-02-08 02:55:53,053 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675824953053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675824953053"}]},"ts":"1675824953053"} 2023-02-08 02:55:53,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-08 02:55:53,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure c839d1c01914d7e4669d3cac724aca6b, server=jenkins-hbase12.apache.org,37605,1675824833879 in 216 msec 2023-02-08 02:55:53,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-08 02:55:53,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c839d1c01914d7e4669d3cac724aca6b, UNASSIGN in 227 msec 2023-02-08 02:55:53,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675824953058"}]},"ts":"1675824953058"} 2023-02-08 02:55:53,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-08 02:55:53,069 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-08 02:55:53,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 297 msec 2023-02-08 02:55:53,078 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:53,078 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:53,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-08 02:55:53,079 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-08 02:55:53,079 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:53,079 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:53,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.HMaster$5(2215): Client=jenkins//136.243.104.168 delete testSplitTableRegion 2023-02-08 02:55:53,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,082 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,082 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:53,082 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:53,082 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 02:55:53,086 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,086 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:53,086 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:53,089 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/recovered.edits] 2023-02-08 02:55:53,089 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/recovered.edits] 2023-02-08 02:55:53,089 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2, FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/recovered.edits] 2023-02-08 02:55:53,092 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/28d512ac2b99449b952e8124af3fe4d8 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf1/28d512ac2b99449b952e8124af3fe4d8 2023-02-08 02:55:53,092 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d235df02936041d39dd0420711abd684 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf1/d235df02936041d39dd0420711abd684 2023-02-08 02:55:53,094 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/07be6d207ea84107b9438314224b9ba1 2023-02-08 02:55:53,095 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/14078b16fc1a4bd5b05ec3c32135e0e4 2023-02-08 02:55:53,095 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8ac6b1b53a714a21ab6c5eb926df5526 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/cf2/8ac6b1b53a714a21ab6c5eb926df5526 2023-02-08 02:55:53,095 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/047b347bb0a645ffa074829582fa23e6 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/cf2/047b347bb0a645ffa074829582fa23e6 2023-02-08 02:55:53,097 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/69f942e90abb4c42888d6c03d0e212cf 2023-02-08 02:55:53,098 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/a394bc9df218479d828df011e8208744 2023-02-08 02:55:53,098 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b/recovered.edits/90.seqid 2023-02-08 02:55:53,099 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/recovered.edits/90.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc/recovered.edits/90.seqid 2023-02-08 02:55:53,099 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/c839d1c01914d7e4669d3cac724aca6b 2023-02-08 02:55:53,099 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/18cda76e13e82f572bb3aeb083809cfc 2023-02-08 02:55:53,099 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/d56f4934dfcb444f948258a1bf772b80 2023-02-08 02:55:53,100 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f2b8b7d7bf72479ebe635ffb28ae6e5e 2023-02-08 02:55:53,101 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf1/f34c1c97f4b5460ea6db3931b2d32dd9 2023-02-08 02:55:53,104 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/019f17c219184d5d82906805131e5a7d 2023-02-08 02:55:53,105 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/03b9ce359ce5430fbc4d34d9379f3dab 2023-02-08 02:55:53,106 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/28124afe734849d78fbea306b6486b2c 2023-02-08 02:55:53,107 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/32e5a593f41b4d9d998c09cca7533dc2 2023-02-08 02:55:53,108 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/88f35dc1ef7b4656b4fdbc4080847668 2023-02-08 02:55:53,109 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/8e5cc008adc24c53b6ef74cf48dda992 2023-02-08 02:55:53,111 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414 to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/cf2/c8a273b718c94f7f94f39b97d8f81414 2023-02-08 02:55:53,114 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/recovered.edits/85.seqid to hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/archive/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01/recovered.edits/85.seqid 2023-02-08 02:55:53,115 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/.tmp/data/default/testSplitTableRegion/3483493d94d3b49573f86ca8f1b07c01 2023-02-08 02:55:53,115 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-08 02:55:53,117 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,119 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-08 02:55:53,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-08 02:55:53,122 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,122 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-08 02:55:53,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824953122"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:53,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824953122"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:53,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675824953122"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:53,124 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 02:55:53,124 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3483493d94d3b49573f86ca8f1b07c01, NAME => 'testSplitTableRegion,,1675824948151.3483493d94d3b49573f86ca8f1b07c01.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c839d1c01914d7e4669d3cac724aca6b, NAME => 'testSplitTableRegion,,1675824951558.c839d1c01914d7e4669d3cac724aca6b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 18cda76e13e82f572bb3aeb083809cfc, NAME => 'testSplitTableRegion,41,1675824951558.18cda76e13e82f572bb3aeb083809cfc.', STARTKEY => '41', ENDKEY => ''}] 2023-02-08 02:55:53,124 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-08 02:55:53,124 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675824953124"}]},"ts":"9223372036854775807"} 2023-02-08 02:55:53,126 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-08 02:55:53,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-08 02:55:53,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 64 msec 2023-02-08 02:55:53,183 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 02:55:53,183 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 02:55:53,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39013] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-08 02:55:53,184 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-08 02:55:53,196 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374 (was 372) - Thread LEAK? -, OpenFileDescriptor=824 (was 839), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 224), ProcessCount=172 (was 172), AvailableMemoryMB=3363 (was 3461) 2023-02-08 02:55:53,196 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-08 02:55:53,197 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 02:55:53,197 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x140ff023 to 127.0.0.1:65244 2023-02-08 02:55:53,197 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,198 DEBUG [Listener at localhost.localdomain/45781] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-08 02:55:53,198 DEBUG [Listener at localhost.localdomain/45781] util.JVMClusterUtil(257): Found active master hash=1487666840, stopped=false 2023-02-08 02:55:53,198 INFO [Listener at localhost.localdomain/45781] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:55:53,206 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 02:55:53,206 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 02:55:53,206 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:55:53,206 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 02:55:53,206 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 02:55:53,206 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 02:55:53,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:55:53,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:55:53,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:55:53,207 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74ecf107 to 127.0.0.1:65244 2023-02-08 02:55:53,207 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 02:55:53,207 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,46783,1675824833788' ***** 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,36075,1675824833838' ***** 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,37605,1675824833879' ***** 2023-02-08 02:55:53,208 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 02:55:53,208 INFO [RS:0;jenkins-hbase12:46783] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 02:55:53,208 INFO [RS:2;jenkins-hbase12:37605] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 02:55:53,208 INFO [RS:1;jenkins-hbase12:36075] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 02:55:53,209 INFO [RS:0;jenkins-hbase12:46783] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 02:55:53,209 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 02:55:53,209 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 02:55:53,209 INFO [RS:0;jenkins-hbase12:46783] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 02:55:53,209 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 02:55:53,209 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:53,209 INFO [RS:1;jenkins-hbase12:36075] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 02:55:53,209 INFO [RS:2;jenkins-hbase12:37605] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 02:55:53,209 INFO [RS:1;jenkins-hbase12:36075] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 02:55:53,209 DEBUG [RS:0;jenkins-hbase12:46783] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66db461f to 127.0.0.1:65244 2023-02-08 02:55:53,209 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:53,209 INFO [RS:2;jenkins-hbase12:37605] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 02:55:53,210 DEBUG [RS:1;jenkins-hbase12:36075] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17fe48f2 to 127.0.0.1:65244 2023-02-08 02:55:53,210 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(3304): Received CLOSE for 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:55:53,209 DEBUG [RS:0;jenkins-hbase12:46783] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,210 DEBUG [RS:1;jenkins-hbase12:36075] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,210 INFO [RS:0;jenkins-hbase12:46783] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 02:55:53,210 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,36075,1675824833838; all regions closed. 2023-02-08 02:55:53,210 INFO [RS:0;jenkins-hbase12:46783] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 02:55:53,210 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:53,210 INFO [RS:0;jenkins-hbase12:46783] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 02:55:53,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1dababf94e1d1ac62f98f642609de9f5, disabling compactions & flushes 2023-02-08 02:55:53,210 DEBUG [RS:2;jenkins-hbase12:37605] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2afdd777 to 127.0.0.1:65244 2023-02-08 02:55:53,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:55:53,210 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-08 02:55:53,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:55:53,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. after waiting 0 ms 2023-02-08 02:55:53,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:55:53,210 DEBUG [RS:2;jenkins-hbase12:37605] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1dababf94e1d1ac62f98f642609de9f5 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-08 02:55:53,211 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-08 02:55:53,211 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1479): Online Regions={1dababf94e1d1ac62f98f642609de9f5=hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5.} 2023-02-08 02:55:53,211 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-08 02:55:53,211 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-08 02:55:53,211 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 02:55:53,211 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 02:55:53,211 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 02:55:53,211 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 02:55:53,212 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 02:55:53,212 DEBUG [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-08 02:55:53,212 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-08 02:55:53,212 DEBUG [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1505): Waiting on 1dababf94e1d1ac62f98f642609de9f5 2023-02-08 02:55:53,224 DEBUG [RS:1;jenkins-hbase12:36075] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs 2023-02-08 02:55:53,224 INFO [RS:1;jenkins-hbase12:36075] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C36075%2C1675824833838:(num 1675824835580) 2023-02-08 02:55:53,224 DEBUG [RS:1;jenkins-hbase12:36075] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,224 INFO [RS:1;jenkins-hbase12:36075] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,228 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,228 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,228 INFO [regionserver/jenkins-hbase12:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,229 INFO [RS:1;jenkins-hbase12:36075] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 02:55:53,229 INFO [RS:1;jenkins-hbase12:36075] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 02:55:53,229 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 02:55:53,229 INFO [RS:1;jenkins-hbase12:36075] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 02:55:53,229 INFO [RS:1;jenkins-hbase12:36075] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 02:55:53,230 INFO [RS:1;jenkins-hbase12:36075] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:36075 2023-02-08 02:55:53,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/.tmp/info/a484669727d84c78b17ae8c4c6b8996a 2023-02-08 02:55:53,236 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/info/daf2340faf564cbe804b7b9c4d99ac45 2023-02-08 02:55:53,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/.tmp/info/a484669727d84c78b17ae8c4c6b8996a as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/info/a484669727d84c78b17ae8c4c6b8996a 2023-02-08 02:55:53,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/info/a484669727d84c78b17ae8c4c6b8996a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-08 02:55:53,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1dababf94e1d1ac62f98f642609de9f5 in 32ms, sequenceid=6, compaction requested=false 2023-02-08 02:55:53,243 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for daf2340faf564cbe804b7b9c4d99ac45 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,36075,1675824833838 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,244 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,245 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2c25d33f rejected from java.util.concurrent.ThreadPoolExecutor@281a5263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,245 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,245 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5e815f15 rejected from java.util.concurrent.ThreadPoolExecutor@281a5263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,246 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/namespace/1dababf94e1d1ac62f98f642609de9f5/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-08 02:55:53,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:53,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:55:53,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1dababf94e1d1ac62f98f642609de9f5: Waiting for close lock at 1675824953210Running coprocessor pre-close hooks at 1675824953210Disabling compacts and flushes for region at 1675824953210Disabling writes for close at 1675824953211 (+1 ms)Obtaining lock to block concurrent updates at 1675824953211Preparing flush snapshotting stores in 1dababf94e1d1ac62f98f642609de9f5 at 1675824953211Finished memstore snapshotting hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675824953211Flushing stores of hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. at 1675824953212 (+1 ms)Flushing 1dababf94e1d1ac62f98f642609de9f5/info: creating writer at 1675824953212Flushing 1dababf94e1d1ac62f98f642609de9f5/info: appending metadata at 1675824953218 (+6 ms)Flushing 1dababf94e1d1ac62f98f642609de9f5/info: closing flushed file at 1675824953218Flushing 1dababf94e1d1ac62f98f642609de9f5/info: reopening flushed file at 1675824953238 (+20 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 1dababf94e1d1ac62f98f642609de9f5 in 32ms, sequenceid=6, compaction requested=false at 1675824953243 (+5 ms)Writing region close event to WAL at 1675824953244 (+1 ms)Running coprocessor post-close hooks at 1675824953247 (+3 ms)Closed at 1675824953247 2023-02-08 02:55:53,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675824835986.1dababf94e1d1ac62f98f642609de9f5. 2023-02-08 02:55:53,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/rep_barrier/5a1de4f897884480a965f1ab257d1791 2023-02-08 02:55:53,255 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a1de4f897884480a965f1ab257d1791 2023-02-08 02:55:53,264 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/table/d2eda39adcea4ff1b0b875a3b0f45a5c 2023-02-08 02:55:53,265 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,36075,1675824833838] 2023-02-08 02:55:53,266 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,36075,1675824833838; numProcessing=1 2023-02-08 02:55:53,268 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2eda39adcea4ff1b0b875a3b0f45a5c 2023-02-08 02:55:53,269 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/info/daf2340faf564cbe804b7b9c4d99ac45 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/info/daf2340faf564cbe804b7b9c4d99ac45 2023-02-08 02:55:53,272 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for daf2340faf564cbe804b7b9c4d99ac45 2023-02-08 02:55:53,272 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/info/daf2340faf564cbe804b7b9c4d99ac45, entries=59, sequenceid=210, filesize=11.1 K 2023-02-08 02:55:53,273 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/rep_barrier/5a1de4f897884480a965f1ab257d1791 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/rep_barrier/5a1de4f897884480a965f1ab257d1791 2023-02-08 02:55:53,276 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,36075,1675824833838 already deleted, retry=false 2023-02-08 02:55:53,276 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a1de4f897884480a965f1ab257d1791 2023-02-08 02:55:53,276 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,36075,1675824833838 expired; onlineServers=2 2023-02-08 02:55:53,276 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/rep_barrier/5a1de4f897884480a965f1ab257d1791, entries=24, sequenceid=210, filesize=7.5 K 2023-02-08 02:55:53,277 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/.tmp/table/d2eda39adcea4ff1b0b875a3b0f45a5c as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/table/d2eda39adcea4ff1b0b875a3b0f45a5c 2023-02-08 02:55:53,280 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d2eda39adcea4ff1b0b875a3b0f45a5c 2023-02-08 02:55:53,280 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/table/d2eda39adcea4ff1b0b875a3b0f45a5c, entries=36, sequenceid=210, filesize=8.0 K 2023-02-08 02:55:53,281 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 69ms, sequenceid=210, compaction requested=false 2023-02-08 02:55:53,287 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-08 02:55:53,287 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-08 02:55:53,288 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 02:55:53,288 INFO [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 02:55:53,288 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675824953211Running coprocessor pre-close hooks at 1675824953211Disabling compacts and flushes for region at 1675824953211Disabling writes for close at 1675824953212 (+1 ms)Obtaining lock to block concurrent updates at 1675824953212Preparing flush snapshotting stores in 1588230740 at 1675824953212Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1675824953212Flushing stores of hbase:meta,,1.1588230740 at 1675824953214 (+2 ms)Flushing 1588230740/info: creating writer at 1675824953214Flushing 1588230740/info: appending metadata at 1675824953224 (+10 ms)Flushing 1588230740/info: closing flushed file at 1675824953224Flushing 1588230740/rep_barrier: creating writer at 1675824953244 (+20 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675824953246 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675824953246Flushing 1588230740/table: creating writer at 1675824953255 (+9 ms)Flushing 1588230740/table: appending metadata at 1675824953257 (+2 ms)Flushing 1588230740/table: closing flushed file at 1675824953257Flushing 1588230740/info: reopening flushed file at 1675824953269 (+12 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675824953273 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1675824953277 (+4 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 69ms, sequenceid=210, compaction requested=false at 1675824953281 (+4 ms)Writing region close event to WAL at 1675824953284 (+3 ms)Running coprocessor post-close hooks at 1675824953287 (+3 ms)Closed at 1675824953288 (+1 ms) 2023-02-08 02:55:53,289 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase12:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-08 02:55:53,366 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,366 INFO [RS:1;jenkins-hbase12:36075] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,36075,1675824833838; zookeeper connection closed. 2023-02-08 02:55:53,366 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@264c0b7f rejected from java.util.concurrent.ThreadPoolExecutor@281a5263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,366 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:36075-0x101407d68340002, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,366 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3e7b4551] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3e7b4551 2023-02-08 02:55:53,366 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@599ab90f rejected from java.util.concurrent.ThreadPoolExecutor@281a5263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,412 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,46783,1675824833788; all regions closed. 2023-02-08 02:55:53,412 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,37605,1675824833879; all regions closed. 2023-02-08 02:55:53,414 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-08 02:55:53,415 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-08 02:55:53,419 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/WALs/jenkins-hbase12.apache.org,46783,1675824833788/jenkins-hbase12.apache.org%2C46783%2C1675824833788.meta.1675824835694.meta not finished, retry = 0 2023-02-08 02:55:53,420 DEBUG [RS:2;jenkins-hbase12:37605] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs 2023-02-08 02:55:53,421 INFO [RS:2;jenkins-hbase12:37605] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C37605%2C1675824833879:(num 1675824835580) 2023-02-08 02:55:53,421 DEBUG [RS:2;jenkins-hbase12:37605] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,421 INFO [RS:2;jenkins-hbase12:37605] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,421 INFO [RS:2;jenkins-hbase12:37605] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-08 02:55:53,422 INFO [RS:2;jenkins-hbase12:37605] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 02:55:53,422 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 02:55:53,422 INFO [RS:2;jenkins-hbase12:37605] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 02:55:53,422 INFO [RS:2;jenkins-hbase12:37605] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 02:55:53,423 INFO [RS:2;jenkins-hbase12:37605] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:37605 2023-02-08 02:55:53,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:53,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,434 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@65f8183b rejected from java.util.concurrent.ThreadPoolExecutor@61c7905c[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,37605,1675824833879 2023-02-08 02:55:53,444 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,37605,1675824833879] 2023-02-08 02:55:53,445 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,37605,1675824833879; numProcessing=2 2023-02-08 02:55:53,455 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,37605,1675824833879 already deleted, retry=false 2023-02-08 02:55:53,455 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,37605,1675824833879 expired; onlineServers=1 2023-02-08 02:55:53,491 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-08 02:55:53,491 INFO [regionserver/jenkins-hbase12:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-08 02:55:53,524 DEBUG [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs 2023-02-08 02:55:53,524 INFO [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C46783%2C1675824833788.meta:.meta(num 1675824835694) 2023-02-08 02:55:53,534 DEBUG [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/oldWALs 2023-02-08 02:55:53,534 INFO [RS:0;jenkins-hbase12:46783] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase12.apache.org%2C46783%2C1675824833788:(num 1675824835580) 2023-02-08 02:55:53,534 DEBUG [RS:0;jenkins-hbase12:46783] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,534 INFO [RS:0;jenkins-hbase12:46783] regionserver.LeaseManager(133): Closed leases 2023-02-08 02:55:53,534 INFO [RS:0;jenkins-hbase12:46783] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase12:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-08 02:55:53,535 INFO [regionserver/jenkins-hbase12:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 02:55:53,536 INFO [RS:0;jenkins-hbase12:46783] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:46783 2023-02-08 02:55:53,543 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase12.apache.org,46783,1675824833788 2023-02-08 02:55:53,543 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 02:55:53,543 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4c561c38 rejected from java.util.concurrent.ThreadPoolExecutor@4869816a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,545 INFO [RS:2;jenkins-hbase12:37605] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,37605,1675824833879; zookeeper connection closed. 2023-02-08 02:55:53,545 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@30376b2f rejected from java.util.concurrent.ThreadPoolExecutor@61c7905c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,545 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@d08fe3b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@d08fe3b 2023-02-08 02:55:53,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:37605-0x101407d68340003, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,545 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@48b3b34b rejected from java.util.concurrent.ThreadPoolExecutor@61c7905c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,554 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase12.apache.org,46783,1675824833788] 2023-02-08 02:55:53,554 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase12.apache.org,46783,1675824833788; numProcessing=3 2023-02-08 02:55:53,564 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase12.apache.org,46783,1675824833788 already deleted, retry=false 2023-02-08 02:55:53,564 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase12.apache.org,46783,1675824833788 expired; onlineServers=0 2023-02-08 02:55:53,565 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase12.apache.org,39013,1675824832471' ***** 2023-02-08 02:55:53,565 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-08 02:55:53,566 DEBUG [M:0;jenkins-hbase12:39013] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1996f4a3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase12.apache.org/136.243.104.168:0 2023-02-08 02:55:53,567 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegionServer(1145): stopping server jenkins-hbase12.apache.org,39013,1675824832471 2023-02-08 02:55:53,567 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegionServer(1171): stopping server jenkins-hbase12.apache.org,39013,1675824832471; all regions closed. 2023-02-08 02:55:53,567 DEBUG [M:0;jenkins-hbase12:39013] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 02:55:53,567 DEBUG [M:0;jenkins-hbase12:39013] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-08 02:55:53,567 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-08 02:55:53,567 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675824835127] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.large.0-1675824835127,5,FailOnTimeoutGroup] 2023-02-08 02:55:53,567 DEBUG [master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675824835127] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase12:0:becomeActiveMaster-HFileCleaner.small.0-1675824835127,5,FailOnTimeoutGroup] 2023-02-08 02:55:53,567 DEBUG [M:0;jenkins-hbase12:39013] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-08 02:55:53,568 INFO [M:0;jenkins-hbase12:39013] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-08 02:55:53,568 INFO [M:0;jenkins-hbase12:39013] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-08 02:55:53,568 INFO [M:0;jenkins-hbase12:39013] hbase.ChoreService(369): Chore service for: master/jenkins-hbase12:0 had [] on shutdown 2023-02-08 02:55:53,569 DEBUG [M:0;jenkins-hbase12:39013] master.HMaster(1502): Stopping service threads 2023-02-08 02:55:53,569 INFO [M:0;jenkins-hbase12:39013] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-08 02:55:53,570 ERROR [M:0;jenkins-hbase12:39013] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-08 02:55:53,570 INFO [M:0;jenkins-hbase12:39013] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 02:55:53,570 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-08 02:55:53,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-08 02:55:53,575 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 02:55:53,575 DEBUG [M:0;jenkins-hbase12:39013] zookeeper.ZKUtil(398): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-08 02:55:53,575 WARN [M:0;jenkins-hbase12:39013] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-08 02:55:53,575 INFO [M:0;jenkins-hbase12:39013] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 02:55:53,576 INFO [M:0;jenkins-hbase12:39013] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-08 02:55:53,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 02:55:53,576 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 02:55:53,576 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:55:53,576 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:55:53,576 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 02:55:53,576 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:55:53,576 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.57 KB heapSize=719.55 KB 2023-02-08 02:55:53,592 INFO [M:0;jenkins-hbase12:39013] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.57 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ee54dedc102b4304a93aac66a0455d60 2023-02-08 02:55:53,596 INFO [M:0;jenkins-hbase12:39013] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee54dedc102b4304a93aac66a0455d60 2023-02-08 02:55:53,597 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ee54dedc102b4304a93aac66a0455d60 as hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ee54dedc102b4304a93aac66a0455d60 2023-02-08 02:55:53,599 INFO [M:0;jenkins-hbase12:39013] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee54dedc102b4304a93aac66a0455d60 2023-02-08 02:55:53,600 INFO [M:0;jenkins-hbase12:39013] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36267/user/jenkins/test-data/fdc173b5-8c0e-44cf-eba0-f00aee112621/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ee54dedc102b4304a93aac66a0455d60, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-08 02:55:53,600 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegion(2947): Finished flush of dataSize ~608.57 KB/623179, heapSize ~719.54 KB/736808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=1282, compaction requested=false 2023-02-08 02:55:53,601 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 02:55:53,601 DEBUG [M:0;jenkins-hbase12:39013] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675824953576Disabling compacts and flushes for region at 1675824953576Disabling writes for close at 1675824953576Obtaining lock to block concurrent updates at 1675824953576Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675824953576Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623179, getHeapSize=736808, getOffHeapSize=0, getCellsCount=1499 at 1675824953576Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675824953577 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675824953577Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675824953582 (+5 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675824953582Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675824953597 (+15 ms)Finished flush of dataSize ~608.57 KB/623179, heapSize ~719.54 KB/736808, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=1282, compaction requested=false at 1675824953600 (+3 ms)Writing region close event to WAL at 1675824953601 (+1 ms)Closed at 1675824953601 2023-02-08 02:55:53,604 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 02:55:53,604 INFO [M:0;jenkins-hbase12:39013] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-08 02:55:53,604 INFO [M:0;jenkins-hbase12:39013] ipc.NettyRpcServer(158): Stopping server on /136.243.104.168:39013 2023-02-08 02:55:53,617 DEBUG [M:0;jenkins-hbase12:39013] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase12.apache.org,39013,1675824832471 already deleted, retry=false 2023-02-08 02:55:53,654 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,654 INFO [RS:0;jenkins-hbase12:46783] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,46783,1675824833788; zookeeper connection closed. 2023-02-08 02:55:53,655 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4067fc8e rejected from java.util.concurrent.ThreadPoolExecutor@4869816a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,655 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:46783-0x101407d68340001, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,655 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@587c22f2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@587c22f2 2023-02-08 02:55:53,655 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@384a2b48 rejected from java.util.concurrent.ThreadPoolExecutor@4869816a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,655 INFO [Listener at localhost.localdomain/45781] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-08 02:55:53,754 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,754 INFO [M:0;jenkins-hbase12:39013] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase12.apache.org,39013,1675824832471; zookeeper connection closed. 2023-02-08 02:55:53,755 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2f8ee175 rejected from java.util.concurrent.ThreadPoolExecutor@32b088bb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,755 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:39013-0x101407d68340000, quorum=127.0.0.1:65244, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 02:55:53,755 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@472aa103 rejected from java.util.concurrent.ThreadPoolExecutor@32b088bb[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 02:55:53,756 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 02:55:53,792 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 02:55:53,908 WARN [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-08 02:55:53,908 WARN [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 506934cb-153f-48e7-8277-804b83b91344) service to localhost.localdomain/127.0.0.1:36267 2023-02-08 02:55:53,911 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data5/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:53,912 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data6/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:53,914 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 02:55:53,916 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 02:55:54,026 WARN [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-08 02:55:54,027 WARN [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 5f50a424-dee0-448f-aec2-2d8d7ee67cbb) service to localhost.localdomain/127.0.0.1:36267 2023-02-08 02:55:54,029 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data3/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:54,029 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data4/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:54,030 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 02:55:54,032 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 02:55:54,134 WARN [BP-681140092-136.243.104.168-1675824827679 heartbeating to localhost.localdomain/127.0.0.1:36267] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-681140092-136.243.104.168-1675824827679 (Datanode Uuid 8382d232-5ed5-4e55-9828-0c5294970b5d) service to localhost.localdomain/127.0.0.1:36267 2023-02-08 02:55:54,135 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data1/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:54,136 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/9ea4b951-eec0-8668-bb76-f6a8aa0c6589/cluster_ab71d88c-8abc-0129-26a8-c5e115462085/dfs/data/data2/current/BP-681140092-136.243.104.168-1675824827679] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 02:55:54,161 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-08 02:55:54,348 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-08 02:55:54,582 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1293): Minicluster is down